DISCRETE SERIES WHITTAKER FUNCTIONS ON Spin{2n,2) 



KENJI TANIGUCHI 

' Abstract. Discrete series Whittaker functions on Spin{2n, 2) are studied. 

The dimensions of the space of both algebraic and continuous Whittaker mod- 
elss are exphcitly determined. They are described by a sum of dimensions of 
O ' irreducible representations of Spin{2n — 3,2). Also obtained are the Mellin- 

' Barnes type integral formulas of the Whittaker functions associated with min- 

imal K-tyne vectors. 



E"' ' 1. Introduction 

, Let Ge be a real semisimple Lie group and Ge = KrArN^^ be its Iwasawa 

' decomposition. Let 77 be a one dimensional unitary representation of TVr. Given a 

d . representation tt of Gr, a realization of tt in the induced representation Indj^*?; is 

called a Whittaker model of tt. By this realization, a vector ?; in tt is expressed by 
a function on Gr, which we call the Whittaker function associated with v. 

Mainly from the number theoretical point of view, Takayuki Oda and his col- 
leagues have calculated explicit formulas of Whittaker functions and generalized 
ly-^ ' spherical functions on various low rank groups and of various representations. 

, Besides the number theoretical interest, the theory of Whittaker models are also 

00 ' interesting from the analytic points of view. It is well known that, in the case of 

the principal series of SL{2, M), the Whittaker functions associated with a i^R-type 
' vector is expressed by the classical Whittaker's confluent hypergeometric function. 

In the case of representations of higher rank groups, the image of a Whittaker model 
gives an example of multi-variable confluent hypergeometric functions. Therefore, 
we may expect that we can study such functions by means of representation theory. 

Another interesting aspect is the relationship with the invariants of representa- 
tions. First, let us consider algebraic Whittaker models, namely (g, iirE)-module 
$H ' intertwining operators from a Harish-Chandra (g, iirR)-module tt to Ind^^yy. If the 

one dimensional unitary representation 7] of Ar is non-degenerate (cf.SjSTT)), we can 
judge the existence of non-trivial Whittaker models by the Gelfand-Kirillov dimen- 
sion of TT, and the dimension of the Whittaker models is given by the Bernstein 
degree of it. Secondly, let us consider continuous Whittaker models, namely con- 
tinuous intertwining operators from the G°°-globalization tToo of a (g, AR)-module 
TT to the G°°-induced space C°"-lnd%l7]. We can judge the existence of non-trivial 
continuous Whittaker models by the wave front set of tt. If tt is a discrete series, the 
dimension of continuous Whittaker models are expressed by the Bernstein degree 
and geometric data of nilpotent orbits. These results, due to H. Matumoto ([9], 
[lOj). are summarized in Sj3l 
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The author thinks that we can observe the relationship between the invariants 
of representations and the structure of Whittaker models more clearly when we 
treat Whittaker models of non-quasi-split groups than that of quasi-split groups. 
Suppose Gm is a non-quasi-split real semisimple Lie group. Consider a discrete series 
representation tt which has non-trivial Whittaker models. Then it is observed that 
the degree of an irreducible component of the associated variety corresponds to the 
dimension of the solution space of one system of differential equation. It is also 
observed that the multiplicity of the associated cycle corresponds to the dimension 
of the right Zm, ('?)-niodule structure of the solution space of the gradient type 
differential-difference equation T>^ ^<j) — 0. (For the definition of this equation, see 
§3.4|) . Here, MTsi{ri) is the centralizer of 77 in Mth = Zkji{Ar). This observation is 
obtained in a former paper [13' of the author's. He expects that such correspondence 
holds for general higher rank group cases, and hopes to explain it in a natural way. 

Until now, there are not so many concrete examples of non-ciuasi-split cases. 
For such reasons, we investigates the case when Gm is the non-quasi-split group 
Spin{2n, 2), tt is a discrete series of Gr, and is a non-degenerate character of A'r 
in this paper. This setting is the same as in [B], except for the group Gr. Since 
Spin{4:, 2) ~ SU{2, 2), this paper is a generalization of [B]. 

The sections are organized as follows. In fj^l we briefly review the structure 
of Spin{2n, 2) and parametrize its discrete series representations. The set of dis- 
crete series representations of Spin(2n,2) is divided into 2n -I- 2 parts m — 
l,...,n-|-l. ^is mainly devoted to the presentation of general theory. We review 
H. Matumoto's results on the existence and the dimension of Whittaker models in 
§3.11 J. T. Chang's results on the associated cycles of discrete series in §3.21 and H. 
Yamashita's results on the realization of algebraic Whittaker models of discrete se- 
ries in In we apply Chang's theory to our Spin{2n, 2) case. As a result, 
combined with Theorem 15.131 we obtain the dimension of the space of algebraic 
Whittaker models: 

Theorem 1.1 fCorollarv l3.61 Theorem 13. 8p . Suppose Gr — Spin{2n,2). 

(1) The discrete series tta has non-trivial algebraic Whittaker models if and 
only «/ A g Sm,±, m = 2, . . . ,n. 

(2) The dimension of the space of algebraic Whittaker models of tta, A e S„i,±, 
m — 2, . . . ,n, is 

rSpin(2n-3,C) 



E 



Ain>Ai^>A„i4_i>--->A^_X>/.t^_^>|A„ 



In 21 write down the gradient type differential-difference equations, which 
describe discrete series Whittaker functions, by using Gelfand-Tsetlin basis of the 
minimal Kj^-type. After that, we put them in order. Solving such differential- 
difference equations reduces to getting a coefficient function of some special minimal 
i^R-type vector, which we call "corner vector" . This reduction is discussed in ii4.4[ 
and the result is as follows. 

Theorem 1.2 (Theorem I4.23p . Let Q^_i be the Gelfand-Tsetlin pattern defined 
in Definition \4-. 22\ Let (j){a) — X]QeGT(A) ^(Q; a)Q be a solution of the differential- 
difference equation T>~^ ~ 0, which describes an algebraic Whittaker model of 
the discrete series tt^, A G S,„^-i-, m ~ 2,...,n. Ifc{Q^_i;a) is known, then it 
determines all the c{Q; a) for Q e GT(A) containing the same q2n-4 po.rt as Qn-i- 
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By this theorem, we may restrict om' interest to determine the coefficient function 
c(Q^_i;a). In ^ we deduce a system of differential equations satisfied by the 
coefficient function c(Q; a) for some special Gelfand-Tsetlin pattern Q. The set 
of such special patterns contains Qn-i- Also obtained are the Mellin-Barnes type 
integral expression of these functions. In order to present these results briefly, we 
use the notation defined in §S1[S] See (|4.2ip for the definition of KQ{m'), and see 
ij5.2l for the definition of Uj, f3j, tj, n{Q,'m';a). 

Theorem 1.3 fProposition 15.81 Proposition 15. 9p . Assume Q e GT(A) satisfies 
Km . (j4:23| and (l4^ . 

(1) Define f{Q,m';t) := n{Q,m';a)^^c{Q;a). Then, f{Q,m';t) is a solution 
of 

idl-dl-tl)fiQ,m'-t)^0, 

\ l[{dt2 + c^p) + f{dt, - dt2) X{{dt, + f3p) \ /(Q, m';t) = 0. 

U=l ^ P=3 J 

(2) Let Cj be a loop starting and ending at +oo, crossing the real axis at —aj — 
1 < s < —Oij, and encircling all poles of T{~ap ~ s), p = j, . . . , N2, once 
in the negative direction, hut none of the poles of r(/3p + s), p — 3, . . . , j. 
Define 

(f,^{Q,m';t)\_^J_ f Uplin-»P-^) \ tlK_^{t^) \ 
l//(Q,™';i)J ■ 27:iJc,Yl^^^r{l-l3,-s)\i-t2ri-siti)} ' 

and, for j — 2, . . . , N2, define 

r/f(Q,m';i)1 
\//(Q,m';t) / 

Here, K^{z) and Iu{z) are modified Bessel functions, and l = These 
integrals absolutely converge in \ ({ti = 0} U {^2 = 0}), and they form a 
basis of the solution space of the system of differential equations in (1). 

An interesting fact is that not all of these functions generates a solution of 
the whole differential-difference equation T)^ ^(f) — 0. In ^ ^5.31 we construct shift 
operators satisfied by the functions f^^, fj (Propositions l5.ini I5.12p . By these shift 
operators, we know that only 4 in the 2N2 functions f^ , fj do generate solutions 
of the whole differential-difference equations. 

Theorem 1.4 (Theorem 15. 13|) . Let A e S,„ ±, m = 2, . . . ,n. Then the functions 
f^{Q^_^,n~ l',t), with j = 1,2, L — K,L, completely determines the solutions of 

Some of the above solutions may define a continuous intertwining operator from 
the C°°-globalization (7rX)oo to C°°(Gk/7Vr; ry). 

Theorem 1.5 (Theorem l5.15p . Let A e S„i,±, m = 2, . . . ,n. Suppose the character 
rj o/TVjR is the one defined by (|4.6p . IfTV^ > 0, then the dimension of the space of 

3 



continuous intertwining operators is 



J2 ^^ySp^n(2n-3,C) 



Each continuous intertwining operator corresponds to defined in Theorem I j. 31 
(2). On the other hand, if < 0, then this space is zero. 

When Gr — Spin{2n + 1, 2), we have similar resuhs on the Whittaker models of 
discrete series representations. Details are discussed elsewhere about this case. 

Before ending the introduction, we define some notation. Suppose is a real 
Lie group. Write f)o the Lie algebra of Ht^, f) the complexification of fio and H a 
complexification of iJR. This notation will be applied to groups denoted by other 
Roman letters in the same way without comment. For two integers a < b, let [a, 6] 
be the interval {x G Z|a < x < b}. The imaginary unit is denoted by t. 

2. The group Spin{2n, 2) and its discrete series 

2.1. Structure of Spin(2n,2). Let Gr = Spin{2n,2) C Spiji{2n + 2,C) [n > 
2) be the connected two- fold linear cover of 5*00(2^^,2), whose maximal compact 
subgroup i^R is isomorphic to Spin{2n) x Spin{2). We realize the Lie algebra 
Bo = so(2n, 2) as 

..(O 0\-l ( ^lie Alt2„(M),X22G Alt2(M),\ 

S0(2n,2j-^_^,^^^ X22j Xi2eM2„.2(M) /• 

Let 9X = —^X be a Cartan involution of go and let 0o = ?o + Po the corresponding 
Cartan decomposition. Denote elementary matrices by Eij = {5ki5ij)k,i=i,...,2n+2 
and set Fij :— Eij — Eji. Then 

{Fj,k\l <k<j <2n, or {j, k) = (2n + 2, 2n + 1)} and 

{LF2n+j,k\l < i < 2,1 < fc < 2n} 

are bases of to and po respectively. 
Let 

A-k '■— lF2n+k,2n-2+k, dO := + M.A2, 

and define fj £ ajj by fj{Ak) = Sj_k- Then ao is a maximal abelian subspace of po. 
The restricted root system A (go, cio) is 

A(go,ao) ={±/i±/2,±/i,±/2}. 
Choose a positive system 

A+(go,ao) -{±/i + /2,/i,/2}, 
and denote the corresponding nilpotent subalgebra X]QGA+(go ao)(-0o)" '^O' Here 
(flo)a is the root space corresponding to a root a. One obtains an Iwasawa decom- 
position 

go^fio + cio+no, Gr = i^R^RiVR, 

where Ar = exp ao and A'r — exp no . Let 

Xl F2n-2+j.k + iF2n+jM, (1 < J < 2, 1 < fc < 2n - 2) , 
^/l+/2 -P2ri,2n-1 ~ l'F2n+1.2n + '--F2n+2,2ri-l ~ F2n+2,2n+l, 
X-fl+f2 '■— F2n,2n-1 + l-F2n+1.2n + '--F2n+2,2ri-l + -F2n+2,2ri+l- 
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Then {-^j^. |1 < A: < 2n — 2} is a basis of (flo)/^, and {X±/^+/2} is a basis of 
(0o)±/i+/2- 

2.2. Parameterization of discrete series. Let us now parametrize the discrete 
series of Gr- Take a compact Cartan subalgebra t of g defined by 

n+1 

Tfc := -LF2k,2k-i, t := CTfc. 

k=l 

Let {ej\l < j < n + 1} be the dual of {Tk}. The root systems A := A(g,l) and 
Ac := A{t,t) are 

A = {zbcj ± ej\l <i<j<n + l}, Ac = {iej ±ej\l<i <j < n}, 

respectively. Choose a positive system 

A+ = {ei±ej\l<i<j<n} 

of Ac. There are 2n + 2 positive systems Af _|_, . . . , A^^^ _|_ of A containing A+, 
defined by 

A+ _,_ = A+ U {ci ± e„+i|l < i < m - 1} U {e„+i ± ei|m <i<n}, 

if 1 < m < n + 1, 
A+ _ = A+ U {ei ± e„+i|l < i < m - 1} U {-e„+i ± Cjlm <i<n}, 

if 1 < m < n, 

A++1 _ = A+ U {ei ± e„+i|l < i < n - 1} U {-e„ ± e„+i}. 

The discrete scries representations of Gr are parametrized by Harish-Chandra pa- 
rameters. By definition, the set of Harish-Chandra parameters S+ is 

S"*" = {A G t*|A is A-regular, A^-dominant, and A+p is i^TR-analytieally integral}. 

Here, p is half the sum of positive roots for some positive system of A. Hereafter, 
we denote by tta the discrete series representation corresponding to A e S+ . In our 
case, is divided into 2n + 2 parts: 

n+1 n+1 

S+ = y U y 5„,_, S„,± := {A e S+KA,^) > 0,V/3 e A+,±}. 

m=l m=l 

3. Discrete series Whittaker functions 

3.1. Whittaker models. Let rj : — be a non-degenerate unitary charac- 
ter of A'r. We denote the differential representation no — >■ tR of by the same letter 
7]. "Non-degenerate" means that r] is non-trivial on every root space correspond- 
ing to a simple root of A+(0o,flo)- The space of analytic and smooth Whittaker 

functions are defined by 

A{Gu/Nj,;rj) = {F : Gr C\F{gn) = v{n)-^F{g), for neN^,ge Gr}, 

G°°(Gr/7Vr;77) = {^^ : Gr ^ C\F{gn) = r^{n)~^F{g), for n e 7Vr,5 e Gr}, 

respectively. These are representation spaces of Gr by left translation. 

Let (tt, V) be a representation of Gr or a Harish-Chandra {g, ii'R)-module. A 
realization of {tt,V) in ^(Gr/A^r;?7) is called a Whittaker model of {tt,V). By 
this realization, a vector v € V is expressed by a function on Gr, which we call a 
Whittaker function associated with v. 
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We review some results, due to Matumoto, on the existence and the dimension 
of Whittaker models. 



Theorem 3.1 ([9 ). Let V be an irreducible Harish- Chandra (q, K]^) -module. 

(1) V has a non-trivial Whittaker model if and only if the Gelfand-Kirillov 
dimension Y>'m\V of V is equal to dim no . 

(2) // V has a non-trivial Whittaker model, then the dimension of Whittaker 
models is equal to the Bernstein degree DegV of V . 

Any irreducible Harish-Chandra {g, KR)-modu\e V admits an infinitesimal char- 
acter. Therefore, any smooth Whittaker function ^(v){g), with v € V and ip € 
Homg^if, (y, C°°{Gm/Nr; rj)), is an eigenfunction of the Casimir operator. Since the 
Casimir operator is an elliptic differential operator, is real analytic. Therefore 
we may identify Homj,,K,(y, ^(GR/iV]R; 77)) with Homj,^K„(y^, C°°(G'r/7Vr; 77)), if V 
is irreducible. 

The next problem is to specify the continuous intertwining operators. For a 
Harish-Chandra (g, iirR)-module (tt, V), let (tToo, Voo) be its C°°-globalization. This 
is a Frechet representation of Gr. The space of continuous intertwining operators 
from to C°°iGwL/NM;v) wiU be denoted by Homg;(Ko, G°°(Gr/7Vr; 77)). This 
is a subspace of Homg^if,(y, G°°(Gr/A'r; 77)), and it is isomorphic to the space 

of G^°°-Whittaker vectors (cf 10 ). Here, (Tr^,^^^) is the continuous dual to 
(tToo, Voo) With rcspcct to the ?7(g)-topology. 

The next theorem, also due to Matumoto, presents a condition for the existence 
of non-trivial continuous intertwining operators and the dimension of the space of 
such operators. In order to state his results, we introduce some conventions. Since 
a unitary character tp is determined by the value on the root spaces corresponding 
to simple roots, we may regard tp as an element of t(no/[no, no])* C lQq. Using the 
Killing form, we identify 0q with go- Note that tp is non-degenerate if and only if 
t~^Ad(GR)'0 C 05 — 00 is a principal nilpotent GR-orbit. We denote by 7'(Gr) the 
set of principal nilpotent GR-orbits. The wave front set of V is denoted by WF{V). 
For the definition of wave front set, see [lOj . 

Theorem 3.2 (|10j). Let Gr be a connected real reductive linear Lie group and 
let V be an irreducible Harish-Chandra {g, Kr) -module. Let ip be a non-degenerate 
unitary character on no. 

(1) WhP^iV) is non-zero if and only if ^p € WF(V^). 

(2) Lf V is the Harish-Chandra module of a discrete series representation and 
Tp e WF{V), then 

dimWh^-(l/) = ^^DegV, 

where Wgj. is the little Weyl group of Gr . 

3.2. Chang's results. Let us recall J. T. Chang's papers [5], [3], in which associ- 
ated cycles of discrete series are determined for some cases. 

Let y be a Harish-Chandra (g, _firR)-module. Choose a i^R-stable finite dimen- 
sional generating subspace Vq of V, and define a filtration Vn := C^(g)nVb of V. 
Here {C/(g)„|7i = 0, 1, 2, ... } is the standard filtration of U{q). By this filtration, 
M := giV admits an S{g) ~ grt/(g)-module structure. The associated variety 
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AV{V) of V is the support SuppM C g* of M. Note that AV{V) is a closure of 
finite ii'-orbits on p. Let Xi, . . . , Xk be the irreducible components of AV{V), and 
let TTii be the length of S{Q)<:p- module Mqj. , where *Pi is the minimal prime ideal 
corresponding to the irreducible variety Xi. The associated cycle AC{V) of V is 
the formal sum rmXi. 

Assume that Gr has discrete series. Then g has a compact Cartan subalgebra 
let. Let b = t + ii be a Borel subalgebra of g, and let i? C G be the corresponding 
Borel subgroup. This Borel determines a positive root system A+ of A(g, t) so 
that li corresponds to negative roots. Let A+ C A+ be the set of compact positive 
roots and denote p = 5 J2aGA+ o:, Pc = ^ J2aeAt = P - Pc- We may and do 

regard b as a point on the flag variety X :— G/B. By the choice of b, the X-orbit 
Z := if • b ~ K/KC] B is closed. Here, "dot" means the adjoint action. 

Suppose A S S+ is A+-dominant. Let r be the character of T := expt with 
dr = A — p. This character gives rise to a ii'-homogeneous line bundle on Z, 
and we denote by £a-p its sheaf of local sections on Z. Let j : Z — > A be 
the embedding map, and let j+(£A_p) be the direct image in the category of V- 
modules. It is well known that the global section space r{X, j+{£\-p)) realizes 
the Harish-Chandra module of tta. Note that the lowest ATR-type sheaf for tta is 
Ca-p ® detj\fz\x — ^A~p^+p„ , where J^z\x is the normal sheaf of Z in X. 

Let T*X c:i G Xb (g/b)* — G x b u be the cotangent bundle on X. Here, we 
identify (g/b)* with ii by a fixed invariant bilinear form on g. By this identification 
and TZ ~ K Xkhb (V^ ^1 b), the conormal bundle T^X is isomorphic to K x^nB 
(linp). For each point x G A', representing a Borel subalgebra b^, the fiber T*X in 
T*X is given by (g/bx)* C g*. The moment map 7 : T*X — )• g* is given by, on each 
fiber, the inclusion (g/b^)* ^ Q* ■ For the discrete series tta, the associated variety 
,4V (tta) is a closure of a single A'-orbit on p, and it coincides with the moment map 
image 7(T^A) ~ AT • (li n p). Therefore, the associated cycle AC{'ka) of tta is an 
integral multiple of ^{T^X). 

Theorem 3.3 (jSj). Lei^ he a generic point o/iinp C ^{T^X). Then the associated 
cycle of the discrete series representation tta is 

ACiTTA) ^ dimH°{j-\0,^A~P.+pJ-r-Hi))l{nX). 

Under some condition, the explicit formula of this coefficient can be obtained. 
Let NK{^,n D p) := {k e K\k • ^ e u n p}. Then it is not hard to show that 
7-1(5) ~ NKi£„ur]p)~^ -b C Z. Note that Nxi^^unp) is not a group. In order to 
calculate Nk{£,, u n p), let us consider the following groups. Let S be the set of all 
compact simple roots, (S) the root system generated by 5*. Let [ := t + X]Qg{s) 3a, 
and let q = I + 6 D b be the parabolic subalgebra whose Levi part is [. The analytic 
subgroups with Lie algebras [, q are denoted by A, Q respectively. 

Theorem 3.4 ( 3 ). Let K{£^) be the centralizer of ^ in K and K{£^)^ be the identity 
component of the reductive part of K(^). If 

(3.1) Nk{^, u n p) = (if n Q)KiO = {Kn Q)K{Ol 

then 

H'^{^-\0,CA-p^+P.U-Hi)) ^ Coh-Indt^[|5nQ iK^lnQ Va-7.+p.- 
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Here Coh-Ind is the cohomological induction, i.e. the operation which makes the 
irreducible representation of the large group with the same highest weight as that of 
the .small group. 

3.3. Spin{2n, 2) case. Let us apply the general theory to our Spin{2n, 2) case. 

Proposition 3.5. (1) If A E '^i,±, then Dim(7rA) = 2n. 

(2) // A G Sm^-t, TO = 2, . . . , n, then Dim(7rA) = 4n — 2 = dim no . 

(3) // A e then Dini(7rA) = 4n — 3. 

Proof. Let Xa G 0q be a non-zero root vector for a G A(g, t). For each case, we 
can choose 

{(1) -'^±e„Te„ + i + -'^Te„Te„ + l 
(3) -'^'=(=e„±e„ + i + X_e„_i^e„+i 

as a generic point of li n p. It is not hard to calculate the centralizer K{^), and 
finally we obtain Dim(7rA) = dim{K ■ ^) = dimi^T — diniA'(^) = 2n, An — 2 and 
An — 3, respectively. □ 

Corollary 3.6. The discrete series tt\ has a non-trivial algebraic Whittaker model 
if and only if A £ 'E,m,±, m = 2, . . . ,n. 

Chang has shown that the condition p.ip is satisfied if Gr is a connected real 
rank one group. It is also satisfied in our case. 

Proposition 3.7. The condition p.ip is satisfied when A G S„j^-i-, m = 2, . . . ,n. 
Proof. Suppose A G 'E.m,±, m = 2, . . . , n. In this case, 

m — 1 n 



We choose a generic point ^ of n n p as in the proof of Proposition [3T5] For this ^, 
K{^)r ~ {±1} X Spin(2n - 3, C), 

K{^) = K{^)re^p ^^(aiX_ei+ei +biX^ei~ei) +a„(X_ei+e„ +X^ei-e„)^ , 

where A"q,'s are appropriately chosen non-zero root vectors. For A G Sm,±, the 
parabolic subgroup KCiQ corresponds to the set of simple roots {ei — 62, . . . , em-2 — 
Cm-i, em - Cm+i, ■ ■■ , e„_i - e„, e„_i -I- e„}. Let Qi D K Ci B he the parabolic 
subgroup of K, whose Levi subalgebra corresponds to the set of simple roots {e2 — 
63, ... , e„_i — e„, e„_i -|- e„}. Note that Qi contains Let k = qwqi be the 

Bruhat decomposition of fc G Nk{^, fl H p) with respect to {K n Q)\K/Qi, where 

qeKnQ,we {&,n-i x (©„-™+i x zr™))\e„ x zrV(6n-i X zr"), 91 G Qi- 

Since K CiQ normalizes uHp, fc-^GuHpis equivalent to qi • ^ G w^-^{u n p). By 
direct computation, this is true only if w = e, and we can show that, if gi • ^ G uHp, 
then gi G (if n Q)K(^). Thus we get A^k(C,u n p) C (i^ n Q)K{^). Since the 
inverse inclusion is trivial, we get Nk{^, u ^ p) = {K C] Q)K{S). Finally, since each 
connected component of K{£) meets exp t C KC\Q and the unipotent radical of K{S) 
is contained in X n Q, we know Nk{£., u n p) = {K n Q)K {£) = {K n Q)K{^)°. □ 



Let Vj^ be the irreducible finite dimensional representation of a reductive group 
F with the highest weight ly. For A G S^^, m — 2,...,n, let A — (A;A„_(_i) = 
(Ai, . . . , A„; A„+i) = A — + Pn be the Blattner parameter of tta- Since 

if n Q ~ GL{m - 1, C) X Spin(2{n - m + 1), C) x 5pm(2, C) 
X (unipotent radical), 
n Q ~ GL{m - 2, C) X Spin{2n - 2m + 1, C) k (unipotent radical), 

we have 

yKnQ _ yGL{m-l,C) ^ ^Spm(2(n-m+l),C) ^ t/S'P*"(2,C) 
A ~ (Ai,...,A„_i) (A,„,...,A„) A„ + i ' 



Res I ^'''^ T/^'"^ 



e-j^GL(m-2,C) ^ ySpin{2n-2m+l,C) 



Coh-Tnd t"^^-"-- Res r'^'^ 

(Ml,...,/^m_2:/Ji,...,M„_„) 



-^l>Ml>A2>■■■>A^_2>MTT^-2>-^T7^-l 



Theorem 3.8. T/ie Bernstein degree o/tta, A S ^m.±, m = 2, . . . , n, is 

^5pm(2n-3,C) 
(/ii,...,/i„_2,M'i:---:A'^_,„)' 



C J2 dim y^f 



A^>^^>A2^---^Aj^ 2 — 2— A^ji 1 

where C is a general constant independent of A. 

3.4. Realization of Whittaker functions. Embedding of a discrete series into an 
induced representation is realized by gradient type differential-difference equations. 
We review Yamashita's results (cf. [TB]). 

Let 77 be a non-degenerate unitary character of N^. For a finite dimensional 
representation {T,Vr) of Kt^, define 

CTiKwL\Gm/Nu;v) 
~ {F : Gk ^ Vr\Fikgn) = r^{ny\{k)F{g), for n G A^r, g G Gr, A: G J^r}. 

Let, as before, A = A — pc + pn be the Blattner parameter corresponding to a 
Harish-Chandra parameter A. Let (T-j^jV^) be the irreducible finite dimensional 
representation of Kr with the highest weight A. Let (Ad,p) be the adjoint repre- 
sentation of Kr on p. 

Fix an invariant bilinear form ( , ) on go and choose an orthonormal basis {Xi} 
of pQ. Define a differential-difference operator ^ by 

V3,^^ : G,^(ifR\GM/A^M;^) ^ G,°°^Ad(i^M\GR/^R; ry), 



Here, Ljsf . is the left translation. 



Let A+ be the set of non-compact roots a with (a, A) > 0. Then the irre- 
ducible decomposition of Ad is ffiQ,gA+u(-A+)™"'''A+a' "^ct G {0,1}. Let 
:= ®Q,gA+™-"'''A-Q ^® negative part and let pr~ : ® Ad be 
the natural projection. Define a differential-difference operator I?^ ^ by 

Theorem 3.9 f|16)). Let tt^ 6e </ie rfwa/ Harish- Chandra module of tta. // the 

Blattner A o/ tta is far from the walls, then 

Hom,,K.(7rX,C°°(GM/7VR;r;)) ^ Ker(I?3^ ,,). 

Remark 3.10. Suppose Gr = Spin{2n, 2) and A = X^I^Jl^ M^i. The Harish-Chandra 
parameter of tt^ is A* := X]r=i ^i^i + (— l)"A„e„ — A„+ie„+i. Therefore, if A e 
Sm -t and m < n, then A* € S^^ip. The Blattner parameter A corresponding to 
A = YJi=i ^i^i G S,„^±, m < n, is 

m— 1 n 

A = ^ (Aj - n -f i + l)ej + ^ (Aj - n + i)ej + (A„+i ± (n - m + l))e„+i. 

2—1 i—m 

Especially, the n-th components A„, A* of the Blattner parameters of tta, tt^ are A„, 
(— 1)"A„, respectively. Therefore, if A G 5„i^±, m = 2, . . . ,n, then the Bernstein 
degrees of tta and tt^ are identical because of Theorem 13.81 

4. Radial part of V-^ ^ 

In this section, we write down the differential-difference equation T)^ ^^cj) — 
explicitly. After that, we reduce our computation to getting a coefficient function 
of some special vector. 

4.1. Irreducible decomposition of tensor product representation. The Lie 

algebra t is isomorphic to so(2n, C)®so(2, C) and, as a vector space, p is isomorphic 
to the matrix space M2n,2{'C). The adjoint representation (ad,p) of { on p is 

so(2n, C) ® so(2, C) rv Af2„,2(C), 

{A, B)-X = AX - XB, for A G so(2n, C), B € so(2, C), X € M2„,2(C). 

Let (r*^, C'') be the natural representation of so(fc, C) on C'''. By the identification 
C^" (g) C2 ~ M2„,2(C), u(g>v^ u*v, 

(ad,p)~ (T2",C2")H(r2,C2). 

Note that ti^2n+i.j G P corresponds to the vector H vf e r^" Kl r^, where 

i 

= '(0, . . . 0, 1, 0, . . . , 0) is a standard basis of C*"'. 

We realize the representation rs^ of Xr by using the Gclfand-Tsetlin basis. 

Definition 4.1. Let A = (Ai, . . . , A„) be a dominant integral weight of Spin{2n). 
A (X-)Gelfand-Tsetlin pattern is a set of vectors Q — (qi, . . . , q2n-i) such that 

(1) = {qi,i,Qi,2, ■ ■ ■ ,gi,[(i+i)/2j )• 

(2) The numbers qij are all integers or all half integers. 

(3) (721+1 J > q2ij > fe+ij+i, for any j = 1, . . . , i - 1. 

(4) q2i+i,i > q2i.i > |g2i+i,i+i|- 

(5) q2^,j > q2i-i,j > q2i,j+i , for any j = 1, . . . , j - 1. 
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(6) q2i,i > q2i-i,i > -q2i.i- 

(7) q2n-l,j = Aj. 

Here, [aj is the largest integer not greater than a. The set of all A-Gelfand-Tsetlin 
patterns is denoted by GT{\). 

Notation 4.2. For any set or number * depending on Q e GT{X), we denote it by 
*{Q), if we need to specify Q. For example, QijiQ) is the qij part of Q e GT{X). 

Theorem 4.3 ([5]). Let X be a dominant integral weight of Spin(2n) and let 
(ta, V;f^™^^"^) be the irreducible representation of Spin {2n) with the highest weight 
X. Then GT{X) is a basis o/ (ta, Vjf^*"^^"''). 

The action of elements Fp^g e so(2n) is expressed as follows. For j > 0, let 

hi-lj ■— 921-1 j" + i — i, l2i-l,-j ■= —hi-l,], 

hij ■= q2ij + i + ^ — j, ^"^i.-j ■= ~^2ij + 1, 

and let ^2i,o — 0. Define ap^q{Q) by 



02^-1 J (Q) = sgnj 
for j ~ ±1, . . . , ±i, and 



ni<|fc|<i-l(^2»-l,i + l2i-2,k) I{l<\k\<iihi-l.J + hi.k) 
\ 4 J|i<|fc|<i, (^2i-l,j ■+ /2i-l,fc)(^2i-lj- + ^2i-l,fc + 1) 



a'2i,]{Q) = i2i,]{Q)^ 



ni<ifei<i('2ij + hi-i.k) ni<ifei<i+i('2ij + hi+i, 



\ (4^2i 1 ~ 1) n^^l'^IS* ('2i.j + l2i.k){l2i.j — hi.k) 



for i = 0,±l,...,±i, where e2i,j{Q) is sgnj if j ^ 0, and sgn(g2j-i,jg2i+i.i+i) if 
J=0. 

|fc| 

Let aa^b be the shift operator, sending to + (0, . . . , sgn(6), 0, . . . , 0). For 
notational convenience, we often write t^j := cT2n-3,i'^2n-2,j- 

Theorem 4.4 d^). Under the above notation, the Lie algebra action is expressed 
as 

T\{F2i+iai)Q = ^ a2j-lj(Q)o'2i-ljQ, 

l<\]\<i 

Tx{F2i+2,2i+l)Q = ^ a2i,j{Q)cr2i.,jQ- 

0<\j\<i 

In the following of this paper, we assume that the Blattner parameter A is far 
from the walls. It follows that the numbers Xj satisfy 

(4.1) Ai > A2 > ••• > A„_i > |A„| 

and the differences between adjacent numbers are sufficiently large. 

Let efe = (0, . . . , 1, 0, . . . , 0) e C" and e_fc := -e^. Let 

Prfe ^ C^" ^ 1^+""^'"^ ' for fc = ±1, . . . , ±n 

be the projection operator along the irreducible decomposition 

ySpin(2n) ^ jj-,2n ^ ySpin{2n} 

l<\k\< 
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In order to describe this operator explicitly, we identify Q G GT(A) with the 
Gelfand-Tsetlin pattern (Q, q2n) of a representation of Spin{2n + 1), where q2n '■— 
(Ai + 1, . . . , A„_i + 1, |A„| + 1). Just the same way as in the proof of [SI Proposi- 
tion 4.3] and Lemma 3.1.3], we get the following formulas. 



Lemma 4.5. For Q e GT(A) and k = ±1, . . . , ±n, 

(4.2) pikiQ «) vjl^) = a2„_i,fc((3)0-2n-l,fcQ, 

..2n \ _ ST^ 0,2n~2,j{Q)o-2n-l,k{TojQ) 
hn-2,j — hn-l.k 



0<bl<n-l 

(4.4) prM®vi^_2) 

EV^ (l2n-3,iiQ)o.2n-2,j{TifiQ)a2n-l.kiTi.jQ) 
l<|i|<n-10<b|<«-l - ^2„-2j + l)(/2„-2a - l2n-l,k) 

Remark 4.6. For Q £ GT(A), it is not hard to see that a2i-i,j{Q) = if and 
only if TQ jQ ^ GT{X), and that a2i,j{Q) = if and only if either (i) j ^ and 
<y2i.jQ ^ GT(A) or (h) j = and or g2j+i,t+i = 0. Moreover, 

(1) if j, fc 7^ 0, the coefHcient of TojCr2n-i.fcQ in (|4.3p is non-zero if and only if 
Toja2n-i,kQ G GT(A -I- Cfc), and 

(2) if i,j,k ^ 0, the coefficient of Ti jCr2n-i,kQ in (|4.4p is non-zero if and only 

if n^j(72n-i,kQ e GT(A + e^). 



We know that V^* = y^P^n[zn) ^ y^p;n(z) p ~ C^" H C2. Therefore, the 
irreducible decomposition of V^^- p is 

(4.5) yf^(»p~ ^A+^el^'"' ^ "l^A?+T+l ® ^A+^el''"^ ^■^A?+T-l- 
l<|/c|<n l<|fe|<n 

Since the irreducible representations of Spin(2) are one dimensional, we identify 
Q £ GT(A) with a vector of V^", on which T^+i acts by the scalar A„+i. Such an 

identification is also applied to V^^^J^^^"'^ MV^^^^^l . More precisely, for Q e GT(A), 

regard Qf :— <J2n-i,kQ as a vector in V^jJ™*-^"'' H V^^^^"^^-,', on which T„+i acts by 
the scalar A„+i ± 1. 

Let prj, ^ = pr^, M pr^ be the projection operator from V^^' p to V^f^™''^"'' M 
ySpin(2) j^j^Qj^g |-j^g irreducible decomposition (14. 5p . Then, by normalizing vectors 
appropriately, we have the following explicit formulas from Lemma 14.51 

Lemma 4.7. For Q e GT{\) and k = ±1, . . . , ±n, 

WkAQ ® « ^ "2)) = ci2n-iAQ)Qt^ 



..2n „.2\\ — ^ a.2n-2,j{Q)a2n-l,k{T0jQ) _ ^± 

0<bl<n-l 



prfc.±(Q«) («2«-i ^^'2)) = 2^ — r ^ — TojQk^ 

t2n-2j" — t2n-l,fc 



Wk,±iQ® «-2^V2)) 



i(Q)a2n-2j(Ti,0(3)a2n-l,fc(T'i,iQ) ^± 
l<M<n-10<b|<n-l (^2»-3,* " ^2„-2,, + l)(^2„-2,, - /2„-l,/c) 

pr,^±(Q®(t;f Hi-2)) = Tiprfe_^(g,»(«f S«2)) /or j = 2n, 2n - 1, 27i - 2. 
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The actions of Casimir elements of so{k) C so(2n) on the Gelfand-Tsethn bases 
are as foUows. Let Ck '■— J2i<j<i<k -^ij- Smce this is a constant multiple of the 
Casimir element of so(fc), it acts on an irreducible representation of so(fc) by a 
scalar. Especially it acts on Q S GT(A) by a scalar, since Q is contained in the 
V^^^^'^-'-isotropic subspace. This scalar is calculated in [T31 §5.1]. 

Lemma 4.8. Let 2pk (fc - 2, fc - 4, . . . , /c - 2[fc/2j). For Q e GT(A) and 
= 2, . . . ,2n, 

rA(Cfc)Q = -(|qfc_i|2 + 2(qfe_i,pfe))g. 

4.2. Differential-difference equation ^cj) — 0. Under some appropriate nor- 
malization of the invariant bilinear form ( , ) on Qq, {tF2„+ij |l < i < 2,1 < j < 
2n} forms an orthonormal basis of po- The Iwasawa decompositions of these vectors 
are given by 

iF2n+t.k = X'f^ - F2n-2+t.k, for 1 < i < 2, 1 < fc < 2?1 - 2, 
i'F2„+t,2n-2+i = Ai, for 1 < j < 2, 

l'F2n+2,2n-l = ■^{X-fi+f2 + Xfl+f2) - -P2n,2n-1- 

Let Mr be the centralizer of in K-r. Since Mr acts on no /[no, no], it acts on 
the set of unitary characters of A'r. Therefore, when we calculate Kei{'Dj^ we 
may choose "manageable" non-degenerate unitary character rj in its MR-orbit. Let 
Mr(?7) be the centralizer of rj in Mr. Since MR(7y) acts on Ker(I?^ ^) by the right 
translation, the space of Whittaker models has MR(?7)-module structure. 

As a "manageable" character, we choose rj satisfying 

{^■6) viX^J ^ ['I' '^^''''^ dle,'^""^^' viX-f^+f.) = CV2, ^(X/.+/J = 

with rji > 0,772 7^ 0. Here, we denote the differential of rj by the same symbol 77. 

Because of the Iwasawa decomposition, an element of G^{Kr\Gm/Nt^;t]) is 
determined by its restriction to Ar. Thus, we consider the restriction oi (j) d 
C-(i^K\GR/ivR;77) to Ar. 

Introduce a coordinate in Ar by 

(M>o)^ 3 (ai,a2) exp((log ai)Ai + (log 02)^2) G Ar 
and define di :— aid/dai. The left action of Lie algebra elements on (f> is as follows: 



If X e to, then Lx<j)(a) ^ 

dt 

if X e no, then Lx4>{a) — — 
and for Ai, L^.0(a) — — 



0(exp(-tX)a) = -dT'^{X)(t){a), 

t=o 

0(aexp(-iAd(a"^)X)) = 77(Ad(a"^)X)0(a), 

t=Q 

(t){exp{-tAi)a) = -di(f>{a). 

t=o 
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Since {tF2„+i.j |1 < i < 2,1 < j < 2n} is an orthonormal basis of po, 



2 2n 

i=l j=l 
2 2n-2 

2 

+ Z La,4"E) iF2n+ian-2+i + + )/2~F2„,2„- 1 ® '-F2n+2,2n-l 

1=1 
2 

= - t^2n+i,2n-2+j + t 4> ® '-F2n+1.2ri-2 

^ ai 

2=1 

2a2 

2 2n-3+i 

+ ^ dTl{F2n-2+i,k)(l> ® l'F2n+i,k + l^dT^x(Fn+l)(l> <^ t^2n+l,2n- 

i=l fc=l 

Since 

(dr^^ (g) ad){F2n-2+t.k)'^{v -S) iF2n+i,2n-2+i) 
= dT^{F2n-2+i,k)'^V (Xl ii^2n+i,2ri-2+i 

- 2dT~^{F2„-2+i,k)v (E) iF2n+i,k - V (g) LF2n+t.2n-2+i 

for u e y/^", we have 



rfT3^(f2n-2+i,/c)w ® bF2n+iM 
1 

2 



= -{(drj^ «) l)(i^2«-2+j,fe)^ - (rfT-A ® a.d){F2n-2+iMf - ti^2n+2,2n-2+i)- 



Here, 1 is the trivial representation of {. Since iF2n+i,j corresponds to ti|" Kl vf 
under p ~ C^" Kl and I]fc=i^^' ^2n-2+j,fc = C'2n-2+j - C2„-3+j, we have 



(4.7) 

1 ^ 

„0 =- ^{-29, + {dT~^ (g> l)(C2„-2+2 - C2n-3+z) 
i=l 

- {dr-^ (g) ad)(C2„-2+i - C2n-3+i) - 2n + 3 - z} 

+ ^-</' ® «-2 K +>^T^{<I>^ «-l ^ ^2) + ® « ^ 

ai 2a2 

Let us calculate the projection of ^0 to each irreducible component of y^^*g)p. 
By the identification GT(A) C explained in M.U we write 

0(«) = X! c(Q;a)g. 

Q£GT{\) 
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We need to calculate the projection 

prj,({(dT3^ (g) 1)(C2ti-2+j - C2„-3+i) 
(4.8) - {dT~^ (g) ad)(C2„-2+i - C2n~3+z)}Q «) W2,"-2+») 

= Prfe((rfTA l)(C2«-2+« - C2n~3+i)Q «) 'y2"-2+J 

- rf-TA+e, (C2„-2+j - C2„-3+»)prfe(Q «) v'in^2+t) ■ 

When i — 2, this is a multiple of a2n-i.k{Q)<72n-i,kQ ■ By (14. 2p and Lemma l48l 
its coefficient is 

-(|q2n-lP + 2(q2„-l,P2n)) + (|q2n-2p + 2 (q2„-2 , P2n- 1> ) 

+ (|q2ri-l + CfcP + 2(q2„-l + efe,p2ri)) - (|q2ri-2p + 2 (q2„-2 , P2«- 1) ) 

= 2(sgnfc)A|fc| + f + (sgnfc)(2n - 2\k\) 

= 2?2n-l,fc + 1. 

Analogously, when i = 1, (|4.8p is a linear combination of 

a2n-2,j{Q)a2n-l,k{o'2n~2,jQ) 
l2n-2,j — l2n~l,k 

whose coefficient is 2l2n-2,j- 



C2n-2 jtT2n_l,feQ, 



By these and Lemma mil the projection of g^]) to Kf+"*^"^ M vfj'™^^^^ is 



(4.9) pr,,±(V3,„0) 

= - ^ a2„-i,fc(Q) f 92-'2n-l,fc + ^^- 1tA„+iT y-^j c(Q;a)(5^ 

_|_ ^ a2n-2j (Q)a2n-l,fc('''0j Q) 

QGGT(A) 0<b|<n-l '2n-2,i - ^2^-1,^ 

X f 9i - ^2«-2,j + n-l± j c((5;a)Toj(9fc 



±^ E E E 

QeGT(A) l<|j|<n-l 0<|j|<ri-l 



a2ri-3,i(Q)a2ri-2,j(Ti,oQ)a2n-l,fe('''ijQ) n „± 



('2n-3,i — hn-2,j + l)(^2n-2j" " hn-l,k) 



c{Q;a)T,^jQ^ 

In order to rewrite ()4.9p in a simple form, we need to know the zero points of 
coefficients in the right hand. By Remark 14.61 '^6 have the following lemma: 

Lemma 4.9. Suppose Q G GT(A). 

(1) For k — ±l,...,±n, a2n-i,k{Q) is not zero if and only if (y2n-i.kQ G 
Gr(A + efc). 

(2) For k = ±1, . . . , ±n and j = ±1, . . . , ±{n - I), 

a2n-2j{Q)ci2n-l,k{TojQ) _ a2n-2,j{cr2n-l,kQ)o,2n-l,k{Q) 
hn-2,j — hn-l,k hn~2,j — hn-l.k — 1 

is not zero if and only if To,ja2n~i.kQ G GT{X + efe). 
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(3) For k — ±1, . . . , ±71, j — ±1, . . . , ±{n — 1) and i — ±1, . . . , ±{n — 1), 

a2n~3,i{Q)a2n~2j{TifiQ)a2n-l,k{TijQ) 
{l2n-3,i — hn-2,j + l)(^2n-2j" ^ hn-l.k) 

_ a2n-3,i{T0,jQ)a27i-2,j {(^271-1, kQ)(l2n-l,k{Q) 
(^2n-3,j ~ ^2n-2j)(^2n-2j' ^ hn-l,k — 1) 

is not zero if and only if Tij(72n-i.kQ G GT{X + 6^). 

By this lemma, we can write down (|4.9p in a simple form. 

Proposition 4.10. For k — ±l,...,±n, prfc±(Vj^,j0) ^ is equivalent to the 
following equations. 

(1) IfQe GT(A) and (T2n-i,kQ e GT{\ + ek), then 

d2 - l2n~l,k +n ^ , , 

2a2 / 



1 T A„+i T c(Q;a) 

, U2n-2-j{To,jQ) , , , i_i_^2ai\ , ^ N 

± t > , di + l2n~2,] +n-l± - — c(to,jQ; a) 

n^iTT , hn-2.,j + hn-l.k \ ^^2 / 



0<|i|<n-l 



' l<l«l<n-10<|j|<n-l ('2n-3,.-/2„-2,,)(/2n-2j+/2„-l.fe) 

= 0. 

(2) //g e GT(A), To,j(T2n-i,fcO e GT{\ + e^) and rojQ ^ Gr(A), t/ien 
9i - ^2n-2,j + n - 1 ± -^-'-^ c(Q; a) 

, ?7i a2n-3 -j(Ti,0(3) / r\ \ 
+ i— V ^ c(Ti,o(y;a) 

'^1 i^iTT , t2n-3,i + t2n-2 J 

l<|2|<n— 1 

= 0. 

(3) // g e GT(A), r,,ja2„-i,fcg G Gr(A + e^), T,,,g ^ Gr(A) and T,,og ^ 
GT(A), then 

c{Q;a) = 0. 

Proof. (1) Replace rojg in the second sum of the right hand of (|4.9p with Q and 
after that replace j with — Replace r^^g in the third sum of (|4.9p with Q and 
after that replace {i,j) with {—i,—j). Then we have the above formula. The 
equations in (2) and (3) are obtained in the same way. □ 

We investigate the conditions in Proposition 14.101 Note that, A satisfies (|4.1I) . 
Note also that, if j = 0, then no g G GT{X) satisfies the conditions in Proposi- 
tion|lini(2), (3). 

Lemma 4.11. (1) Q e GT{X) satisfies <T2n-i,kQ ^ GT{\ + efe) if and only if 
one of the following conditions holds. 

(a) k = l. 

(b) fee [2, n - 1] and q2n-2,k-i > ^kF) 

(c) k = -(sgnA„)n. 

(d) k e [-n + 1, -1] and q2n-2,^k < >^-kD 

(e) k = (sgnA„)n and q2n~2,n~i > \K\F> 
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(2) Q E GT(A) satisfies TQja2n-i,kQ ^ GT{\ + e^) and tqjQ ^ GT{X) if and 
only if one of the following conditions holds. 

(a) k e [l,n- 1], j = k, q2n-2,k = Afe and g2ri-3,fe-i > A^D The last 
condition is unnecessary if k = 1 . 

(b) k e [-n + 1,-2], j = -(-fc - 1) = fc + 1, q2n-2-k-i = >^-k and 

'Z2n-3,-fc-l < ^-kD 

(c) fc = -{sgn.Xn)n, j = -(n- 1), g2n-2,«-i = |A„| and |g2n-3,«-i| < 

\Xn\D 

(3) Q e GT{X) satisfies Tija2n-i,kQ & GT{X + ek), njQ ^ GT{X) andn^oQ ^ 
GT{X) if and only if one of the following conditions holds. 

(a) fc e [1, n - 1], i^ j = k, q2n-3M = '72«-2,fc = Afc anrf q2n-4,fc-i > Afcl? 
T/ie last condition is unnecessary if k ^ 1 . 

(b) fc e [-n + 1, -3], i = -(-fc - 2) = fc + 2, j -(-fc - 1) = fc + 1, 

'Z2n-3,-fc-2 = 'Z2ri-2.-fe-l = A_fc BTlrf <Z2n-4,-fc-2 < ^-kD 

(c) fc = -(sgnA„)n, Z = -(n-2), j = -(n- 1), q2n-3,n-2 = g2n-2,n-l = 

|A„| and q2n-i,n-2 < \K\D 

We write down the difFerential-difFerence equation = in the case A S 

E!m,±- In order to write equations briefly, we introduce some notation. 

Notation 4.12. 



Za2 

F±(Q;a) := (2?±(Q) + l2n-2,M^o,jQ\a) 




fn \ - a2n-3.-i(Ti,jQ) 

^1 i^iT^ -, ^2n-3.i — hn~2,j 
Kk <n — 1 



m-2 



I?±(Q) :^d2+Y.l 




2n — l,p 



■2n-2,p + TO — 2 ± A„+i ± 



Ty2ai 
2a2 ' 




By Proposition 14.101 and Lemma I4.11[ we have the following lemma. 




(1) For k e [1, m — 1], if Q E GT(A) satisfies q2n-2,k < ^k, then 
(4.10) 



(02 - hn-i-k + n - l)c{Q;a) + L-^—^ 

Za2 ^ — ' 

0<|j|<n 
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(2) For k e [— n, — 1] U [m, n], if Q G GT{X) satisfies one of the condition 
Lemma \4-.11^ 1 ), then 

(4.11) {d2 - hn-iM + ?^ - 1 ± A„+i ± c{Q- a) 



a2n-2,-j{To,jQ) 

0<b|<n-l 



i-2n-2,j + t2n-l,fc 



= 0. 



(3) For k e [2,m - 1], i/ Q G GT{X) satisfies q2n-2,k-i = Afc and q2n-3,k-i < 
Xk, then 

(4.12) c{Q;a)=0, ^ , "'"''r,^""""^^ c(n,oQ;a)^0. 

l<|i|<n— 1 

(4) Suppose Q G GT{X) satisfies the condition in Lemma \4-. 1 1^ 2) for k £ [— n + 
1, —2] U [m, n — 1] U { — (sgnA„+i)n}. Define j{k) by 

{k if k G [m, n — 1], 

k + 1 i/fce [-n+1,-2], 
— n + 1 i/ fc = — (sgnA„)ri. 

(4.13) 

(Pf (Q) - Z2n-2.j(fe))c(Q;a) + i— Q2n-3 -»(n,oQ) g(-^^^^Q. ^ Q 

"1 l<\t\<n-l '2»-3.« + '2n-2J(fc) 

(5) Suppose fee [-n + 1,-3] U [to,7i - 1] U {-(sgnA„+i)n}. // Q G GT(A) 
satisfies the condition in Lemma \4-ll\ ^3), i.e. if 

92n-3,fc = 92n-2,fc = Afc , (72n-4,*;-i > Afc, w/ien /c € [m, n - 1], 

g2«-3,-fe-2 = 92n-2,-fe-l = X-k, q2n-i-k-2 < A_fc, w/lCn G [-71 +1,-3], 

92n-3,«-2 = '?2n-2,ri-i = | A„ | , g'2«-4,n-2 < |A„|, when k = -(sgnA„)n, 

then 

c{Q-a) = 0. 

By eliminating some terms of the equations in Lemma I4.13i we obtain the fol- 
lowing equations. 

Corollary 4.14. (1) Suppose j e [l,m — 1] satisfies tqjQ e GT{X). Then 
(4.14) (vfiQ) - hn-2.m-i + ^2n-2.j T K+i T ^) c(Q; a) 

4_ ^^72^ ^ °2n-2,-j'(T0j'Q) ni<p<m-l,p^j(^2"-2,j' ~ hn-2,p) 

3'e{3}u[™.,.-i] ripll (^2n-2j' + '2«-l,~p) 

U[-n + l,0] 

X c(Toj'Q;a) 

= 0. 

Here, A„+i = A„+i =p(ri — m + 1) is the {n+l)-.st part of the Harish- Chandra 
parameter A e 
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(2) Suppose j e [— n + 1, — m + 1] U [0, n — 1] . // there exists i G [—n + l,n — 1] 
such that Ti jQ G GT{\), then 



(4.15) (I?2'=(Q) + Z2„-2j)c(Q;a)Tt ^ (j, Q)F/(q. = q. 

i'e{j}u[-m+2,-i] 



(3) Especially, when j — OCthe equation (j4.15l) is 



-1 

(4.16) VtiQ)c{Q;a)Ti ^ B,- (0, g)F/(Q; a) 

j' = -m+2 
np=m^2n-l,pn^l (-'2n-2-p) 

X fpf(Q)c(Q;a) + 3 ^ «2„ 3,-.(r,oQ) ^^^^^Q.^A 
= 



Proof. (I4.14P can be obtained from the equations (|4.10p . applied to fc G [l,m — 1], 
by eliminating c(ro,j'(5; a), j' G [l,m — 1] \ {j}. (|4.15p can be obtained from 
the equations ()4.1ip . apphed to fc G [— n, — 1] U [TO,n], by ehminating Vj'{Q;a), 
j' G hn + 1,-TO + 1] U [0,n- 1] \ {j}. □ 



SupposeQ G GT(A) andro-jQ G G'r(A) for some j G [-n+1, -m+l]U[0, n-1]. 
Replace Q in ()4.15p by tq^-jQ. Then we obtain 



(4-17) T 777^ 7^(2?2^(Q) - ^2„-2,-,)c(ro,-,Q;a) 

+ (2?f (Q) - /2„-2,-,)c(g; a) + c^ V ^H!iZ^:^!iZM^c(r,,oQ; a) 

"1 l<|z|<„-l 2»-3.*+'2«-2,-, 

= 0. 



If Q G GT(A) satisfies 

(1) To -jQ ^ GT(A) and To-ja2n-i-jQ e GT(A + e_j) for some j G [-n + 
1, —to]; or 

(2) To -jQ ^ Gr(A) and to -jCr2n-i,-j-iQ e GT(A-ej+i) for some j G [2,n- 
2]; or 

(3) To -„+iQ ^ Gr(A) and tq -„+io-2n-i -(sgnA„)nQ e GT{\ - (sgnA„)e„), 

then (|4.13p holds for A; such that j{k) = — j (or j(^) = n — 1 in the case (3)). We 
may, and do, regard this equation (|4.13p as a special case of (|4.17p . whose first line 
is zero. 
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Lemma 4.15. // (|4.17p holds for j = ji,...Js and n^oQ e GT(A) for i = 
ii, . . . , is-i, then 



(4 18) ± - I\t=\i^2n-2-j^ +l2n-3,ij 

X (T^tiQ) - hn-2.-3^)c{Ta,-],Q\a) 
n^=l(^2n-2.-jV. + ^2k-3/i J Bj>{j^,TQ^^j^Q) 

X l//(ro,-j;,g;a) 
+ (vfiQ) - ^ ?2„-2.-j, - '2"-3,^„ ) c(g; a) 

\ fJ.= l v=\ ) 

+ t— V ffs — 7^ ^^-7-7 ^ ^ c{n,oQ;a) 

«1 l<|r[^„-l ll/.= l('2n-3,i + i2n-2.-jJ 

= 0. 

Proof. This is obtained from the equations (|4.17l) , apphed to j = ji , . . . , jg , by 
ehminating cijixiQ] a), i = ii, . . . , is~i- □ 

We apply this lemma to some special cases. For m! £ [m — 1, n — 1] and Q E 
GT{X), let 

ifi(m') :={pe [l,m'-l]|Tp^oOeGr(A)}, 

K2im') := [l,m' - l]\Ki{m'), 

Kaim') := {p e [-n + 2, -m']|rp,og e GT(A)}, 

if4(m') := [-n + 2, -m'] \ i^3(m'), 

if5(m') :={p-l|peif3(m')}. 

If p e i^i(m'), then To,-pQ G GT(A) or Ap+i = q2n-2.p > g 2»-3,p > g2«-2,p+i- In 
the latter case, To-pa2n-i-p-iQ G GT{\ — ep+i). Thus (|4.17p holds for j = p 
in each case. If p G Kz{m'), then rp^-p+iQ e GT{\) or q2n-2-p > 92n-3,-p > 
g2n-2, -p+i = A_p+i. In the latter case, ro,-p+icr2n-i,-p+i(3 e GT(A + e_p+i). 
Thus ()4.17p holds for j = p — 1 in each case. 

Let i e Ki{m') U -ft:3(m'), or i = -n + 1 if r_„+i,oQ G GT(A). Define 



/i(m') ii:i(TO') U i^5(™') U {0}, 

/ fm') ^ U {-n + 1} if r^n+iflQ G Gr(A) 

|i^i(m')Ui^3K)U{n-l} if r_„+i,oQ^Gr(A), 

/2(m',i) :=/2(m')\ {«}. 



Then ()4.18p holds for {ji,...,is\ ~ Ii{m') and {ii, . . . , i^-i} = l2{m',i), so the 
following formula is obtained. 
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Corollary 4.16. For i G Ki{m') U Ksim') or i = -n + 1 if T^n+i.oQ e GT{\), 

then 

(4.19) 

?7l a2n-3,-i(T'i,oQ) ni'e/2(m' i)(^2n-3.i — '2n-3,i') 
_ _|_ ^ t ni'e/2(rn',i)('2n-2,-j + ^2n-3,i') 



X (I?^((3) - ^2«-2,-j)c(To,-j(3;a) 

_^ n»'gJ2(m',»)(^2K-2 -j +^2n-3,iO Bj, {j, Tp^^jQ) 

X TAf(ro,-,g;a) 

\ iG-fi(m') i'ehim'.i) ) 



rji a2n-3,-j'('^i',oQ) ni"e/2(m',i)('2«-3,i' - ^2n-3,i") 



ff 71 — 1 ^ c(n',oQ; a). 



Suppose Q e GT(A) satisfies 

(4.20) <?2n-2,j9 = g2n-3,p = g2n-4,p for any p € [1, m - 2]. 
Define 

(4.21) i^6(m') := e [to - U [-n + 1, -m']\TQjQ G GT(A)}. 
For j e KQ{m'), define 

h{m',j) := /i(to') U {-j}. 
Then dHHD holds for {ji, . . . ,jj = Ii{m',j) and{ii, . . . = /2(m'). By 

and the definition of Vj^{Q] a), the term T^// (ro-jiQ) is zero for / e [— m + 2, — 1] 
and j/" G /i(m', j). Thus we obtain the following formula. 

Corollary 4.17. Suppose Q G GT{X) satisfies (|4.20p . //j G Ke{m'), then 
(4.22) 

t nzG/2(r«')(^2n-2j +/2«-3,i) . , ^ N 

^ t nie/2(m')(^2n-2,-j' + '2n-3,i) 



j'e/i(m') ^3'ij'^^0,-j'Q) n/'e/i(m'j)\{j'}('2«-2,-j' - ^2„-2,-i") 
X (^^(Q) - l2n-2-j'HTa-j'Q]a) 

T^fiQ)- X! ^2«-2,-j' - ^ ^2n-3,i c((5;a) 

j'6/i(m'j") ie/2(m') / 

?yi a2n-3,-i('''i,0'9) ni'e/2(m')(^2n-3,i - '2n-3,i') 

+ '-— V ff 7^ — ; ^ c{Ti^oQ;a). 

'^l ,e[_„r;^_„+i, llj'e/i(m'j)(.'2n-3,* + '2«-2 -i'j 

U[m',n-1] 
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4.3. Condition for q2„_4. In this subsection, we deduce necessary conditions for 
q2ri-4 so that the differential-difference equation has a non-trivial solution. In the 
first place, we deduce conditions for q2n-4.k, k [1,to — 2]. 

Lemma 4.18. Suppose fee [1,to — 2]. If Q (z GT{X) satisfies q2n-4.k < ^k+i, then 
c(Q;a)=0. 

Proof. Step 1. Among the Gclfand-Tsctlin patterns with q2n~i,k < Afe+i, there is 
a Q satisfying q2n-2.k = ^k+i and q2n-3.k < ^k+i- By the first equation in (|4.12p . 
c{Q;a) = for this Q. Therefore, by the second equation in ()4.12|) . c{Q;a) ~ 
for Q e GT(A) satisfying q2n-2,k = 92n-3,fc = Afc+i. Thus, c{Q;a) = for all 
Q e GT(A) such that q2n-2,k = Afe+i and g2n-3,fe < Afc+i. 

Step 2. Suppose Q e GT{X) satisfies q2n-i.k < >^k+i and q2n-i,k < q2n-3,k < 
q2n-2,k = Afc+i. In this case, if j' G [m, n— 1]U [— 0]\{fc}, then c{tqj'Q; a) — 
by Step 1, since Tq^'Q satisfies the condition in it. Then by (|4.14l) . applied to this 
Q and j — k, we know that c{Q] a) = for Q e GT{\) satisfying (?2n-2.fc = ^k+i + 1 
and (72ri-3,fc < A^+i. By repeating this discussion, we know that c(Q;a) = for 
Q e GT(A) satisfying Afc > q2n-2,k > A^+i and q2n-3M < >^k+i- 
Step 3. Suppose Q G GT(A) satisfies q2n-2.k = q2n-3,k = >^k+i- By Step 2, 
c{Q;a) = 0. If njQ G GT(A) for ^ {k,k), then it satisfies Afc > q2n-2,k > 
Afc+i and q2n-3.k < '^fc+i; so it satisfies the condition in the conclusion of Step 2. 
It follows that c(TijQ;a) = for ^ {k,k). Then by (|4.15p . applied to this 

Q and j = fc, we have c{Tk.kQ',a) = 0, that is to say c{Q;a) = for Q G GT{X) 
satisfying q2n-4,k < Xk+i and q2n-2.k = q2n-3,k = Afc+i + 1. 

Step 4. Suppose Q G GT(A) satisfies the condition in the conclusion of Step 3. 
Then, by applying the discussion in Step 2 for this Q, we know that c{Q; a) = for 
Q G GT(A) satisfying q2n-3,k < Afc+i + 1. 

By repeating the shift operations as in Step 2 and Step 3, the lemma is shown. □ 

In the second place, we deduce conditions for (72n-4,fe, ^ G [to — 1, n — 2]. 

Lemma 4.19. Suppose fc G [m — 1, rt — 2] . If Q <E GT{\) satisfies q2n-i.k > ^k+i, 
then c{Q; a) = 0. 

Proof. Step 1. Among the Gelfand-Tsctlin patterns with q2n-i.k > Afc+i, there is 
a Q satisfying g2«-3,fe+i = q2n-2,k+i = ^k+i- By Lemma [HTS] (5) , ciQ;a) = for 
this Q. 

Step 2. We know c{Q;a) = for those Q with g2n-3,fc+i = q2n-2,k+i = ^k+i- 
Shift g2n-'3.fe+i downward by using (|4.16p . Then we know that c{Q;a) = for all 
Q G GT(A) satisfying q2n-4.k > Afc+i and Afc+i = g2n-2,fc-i-i > q2n-3,k+i- 
Step 3. Suppose Q G GT(A) satisfies the condition in Step 1. Then by (|4.15|) . 
applied to this Q and j = —k — 1, we know that c{Q;a) — for all Q G GT(A) 
satisfying q2n-4,k > Afc+i and q2n-2,k+i = q2n-3,k+i = Afe+i - 1. 

By repeating the shift operations as in Step 2 and Step 3, the lemma is shown. □ 

Lemma 4.20. If Q E GT{X) satisfies q2n-i,k < ^k+2 for k G [m — l,n — 3], or 

<72n-4,n-2 < | A„ | , then c{Q\ a) = 0. 

Proof. The proof of this lemma is just the same as that of the previous one. In the 
first place, if Q satisfies the above condition and q2n-3,k = q2n-2.k+i = ^k+2 (or 
= |An| if fc = n— 2), we have c((5; a) = by Lemma l4.13l f5). Shift q2n-3.A: upward by 
using (|4.16p . Then we know that c{Q; a) = for all Q G GT{X) satisfying q2n-4,k < 
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\k+2 and (72n-2,fe+i = In the third place, if Q G GT(A) satisfies q2n-3.k = 

g2n-2, fc+i = Afc+2, then shift upward them to q2n^3,k = q2n~2,k+i = >^k+2 + 1 by 
using (j4.15p . By repeating these shift operations, the lemma is shown. □ 

Corollary 4.21. If Q e GT{X) does not satisfy 

A/c > g2n-4./c > Afc+i for k e [l,m-2], 
(4.23) < Afe+i > g2n-4,/c > Aa;+2 for k e [m - l,n - 3], 

^A„_i > (?2ri-4,n-2 > |A„|, 

then c{Q; a) ~ 0. 

4.4. Reduction to the "corner" vectors. Choose a q2n-4 satisfying the con- 
dition (|4.23p . In this subsection, we show that, if c{Qo;a) is known for some 
Qo € GT{X) containing this q2n-4, then it completely determines the other c{Q; a)'s 
for Q G GT(A) containing the same qi, . . . , q2n-4 parts. We call Qo with this prop- 
erty a "corner vector" . 

As will be seen in ^ there is a set of Gelfand-Tsetlin bases Q, including corner 
vectors, such that we can explicitly write down the scalar differential equations 
satisfied by c{Q;a). Such bases Q G GT{X) satisfy the following conditions: For an 

to' G [to — 1, 71 — 1], 



(4.24) 




Ap+i, q2n-3,p ^ q2n-4,p for p G [m - 1 , to' - 1] , 
Ap, q2n-3,p = 92«-4,p-i for p G [to' + 1, n - 1], 

= q2n-3,m' G [A^' + i , g2n-4,m'-l] if ?«' > TO, 

or [Am,Am_i] ifm' = TO-l. 



Definition 4.22. Q^^, G GT(A) is the Gelfand-Tsetlin pattern which satisfies 
!!^m . 623, mM and 92n-2,m' — 92n-3 .rn' 

= A„'+i. Analogously, Q+, G GT(A) 
is the Gelfand-Tsetlin pattern which satisfies (|4.20l) . (I4.23|) . (I4.24p and g2n-2,m' = 

q2n-3,m' = g2n-4,m'-l (or = A^-i if to' = TO - 1). 

Theorem 4.23. If c{Q^_i;a) is known, then it determines all the c{Q;a) for 
Q G GT(A) containing the same qi, . . . , q2n-4 parts as Q^-i- 

Proof. In this proof, we assume all Q G GT(A) contain the same qi, . . . , q2n-4 as 
Qn-i- Note that this q2n-4 satisfies (|4.23p . For simplicity, we prove the case when 
A„ > and m < n. The case when A„ < is proved just in the same way. The case 
when TO = n is also proved just in the same way, and the proof is a little easier. 
For Q — (qi, . . . , q2n-i) G GT{X), define lengths of Q by 

m — 2 n — 2 

||Q||2n-2a := ^ (92n-2J — Aj + i), \\Q\\2n-2,2 ■= ^ (92n-2,i — Aj+i ) , 

j = l j=m-l 
m-2 

||Q||2n-3,l ^ {q2n-3J — '72n-4j), 
J = l 

n — 3 

||Q||2n-3,2 := ^ {q2n-3,j ^ q2n-4,j) 
j=m-l 

+ 92ri-3,n-2 ^ Q'2n-3,n-l ^ |'?2n-2,n-l — '?2n-4,ri-2 1 • 
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Note that, Smce g2«-3,n-2 > <72n-2,n-l > fej-S,™-! and q2n-3.n-2 > q2n-4,n-2 > 

92n-3,n-i, the term 

g2ri-3,n-2 — 92n-3,n-l — |92n-2,n-l — '?2ri-4,n-2| 
— <72n-3,n-2 " '?2n-3,n-l 

- max{g2n-2,ri-l, '72ri-4,n-2} + niin{q2n-2,ri- 1 , '72n-4,n-2 } 

is zero if and only if either (i) q2n-3,n-2 = <?2n-2,n-i and g2n-3,n-i = g2n-4,n-2 or 
(ii) '72n-3,n-2 = g2n-4,n-2 g2n-3,n-i = g2n~2,n-i- Define & partial Order Q' ^ Q in 
GT{X) by 

Q'^Q ^ Q' = Q or Q' -<Q, 

Q'^Q ^ ||Q'||2n-2,l < ll'3l|2n-2,i; 

or ||(5'||2n-2,l = \\Q\\2n-2.1 and WQ'hn-SS < ||Q||2n-3,i; 
or \\Q'\\2n-2,l = ||Q||2«-2,1> llQ'l|2n-3,l = ||Q||2n-3,l 

and \\Q'\\2n-2.2 < ||Q||2«-2,2; 
or ||Q'||2r,-2,l = ||Q||2n-2,l, ||Q'||2n-3,l ||Q||2ri-34> 

\\Q'\\2n-2,2 = ||Q||2n-2,2 and ||Q'||2n-3,2 < ||Q||2n-3,2- 



We will show Theorem 14.231 by induction on this partial order. 
Step 1. If Q satisfies and 



(4.25) 



''?2n-2,p = Ap+i forpe [m- l,n-2], 
92«-3,p = q2n-4,p for p e [m - 1 , n - 3] , 

-^ri-l > 9271-3, n-2 = '?2ri-2,ji-l > <Z2n-4,n-2 
^'?2n-3,ri-l ~ 92)i-4,n-2, 

then the equation (|4.15l) . applied to this Q and j = n — 1, is 
(2?2(Q) + ^2n-2,«-i)c((3; a) 

771 Bn~l{n - 1, Q)a2„_3,-n+2(T^ n, — 2 , n — 1 



Ol ^2n-3,n-2 — ^2n-2,n-l 

0. 



c(r„_2,n-iQ; a) 



If Q satisfies and 

{g2n-2,p = Ap+i, for p e [m - l,n - 2], 
92n-3,p = 92n-4,p for p G [to - 1, n - 2], 
92n-4,n-2 > '?2n-2,ri-l = 92)1-3, n-1 > A„ , 

then the equation (|4.15l) for this Q and j = — (n — 1) is 

(2?2(Q) + ^2ri-2,-ri+l)c((3; fl) 

ryi B_„+i(-n+ 1, Q)a2„-3,n-i(T-,i+i,-„+i(5) 



fll ^2ri-3,-n+l ^ ^2ri-2,-ji+l 

0. 



c(r_„+i,-„+i(5; a) 



By repeating these shift operations, we know that c((5^_]^; a) determines the c{Q] a) 
for Q e GT(A) which satisfies (i) (14201) and (|4:25l) or (ii) (ji:^ and (gSH), in other 
words, for Q which is minimal with respect to the partial order ^. 
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Step 2. Suppose Q £ GT(A) in question satisfies t^ qQ G GT(A) for an i E 
[l,n- 2] U {-n+ 1}. Tlien t,,oQ )~ Q since \\Tt,QQ\\2n-2,p = ||Q||2n-2,p for p = 1,2 
but ||Ti,oQ||2n-3,p > ||Q||2n-3,p for p = 1 Or 2. Consider tlie equation (|4.19l) applied 
to m' = n - 1. If j e /i(n - 1) \ {0} = ii:i(n - 1) C [1, n - 2], then tq -jQ ^ Q 
since ||To,-j(3||2n-2,p < ||Q||2n-2,p for p = 1 or 2. If j e /i(n- 1), j' e [-m + 2, -1] 
and i' G [-n + l,n - 1], then Tj'j'TojQ ^ Q since \\n>j>TojQ\\2n-2,i < ||Q||2n-2,i- 
Moreover, if i' £ [— n + 2, —1] U {n — 1}, then Tj/^qQ ^ Q since ||Ti/,oQ||2n-3,p < 
||Q||2ji-3,p for p = 1 or 2. It foUows that all the c{Q';a)'s appearing in the right 
hand of (|4.19p satisfy Q' ^ Q. Therefore, c{Ti oQ; a) can be expressed as 

cinfiQ: a) = ^ (differential of c{Q'; a)), 
Q'r<Q(-;T..oQ) 

and it is determined by c{Q^_i; a) by the hypothesis of induction. Especially, if 
Q e GT(A) satisfies (I4.20p and q2n-2,p = \+i for any p € [m — 1, n — 2], then we 
know from the result of Step 1 that c(Q^_-^; a) determines c{Q\ a). 
Step 3. Suppose Q e Gr(A) satisfies (|4^ . Let fc G KQ{n - 1). Consider the 
equation ()4.22p applied to m' — n ~ I and j — k. Since j' € Ii{n — 1) implies 
To-j'Q ^ Q and i € [— n + 2, —to + 1] U {n — 1} implies nfiQ -< Q, all the c{Q'; a) 
appearing in the right hand of (I4.22p satisfies Q' ^ Q. Therefore, c{To^kQ', a) 
satisfies 

(4.27) CD^iQ) - l2n-2,k)c{To,kQ\ a)^ J2 (differential of c(g'; a)). 
The equation (j4.14p for j = to — 1 implies 

(4.28) ^ Ak{Q)c{To,kQ]a)= ^ (differential of c(Q'; a)) 

since toj'Q ^ Q for j' G [— n + 2, — m + 1]. 

In order to eliminate extra terms in the left hand of (|4.28p , consider the following 
differential operator. For any m' G [m — 1, n — 1] and j, k G K(,{m'), the fraction in 
the right hand of 

(4.29) n {T^t{Q)-hn-2,p) 

p£Ks(m')\{j} 

_ T\p£K6(m')\{j}i'^2 {Q) ~ hn~2,p) - Y{p(zKe(m')\{3]i^'^n-2,k - ^2k-2,p) 

^2 iQ) ~ hn-2,k 

X {'Dt(.Q)-kn-2,k) 

+ JJ^ {hn-2,k — ^2n-2,p) 

p6K6(m')\{j} 

is a polynomial in 'Df{Q), so it is a differential operator. Note that the last term 
in the right hand is zero if fc 7^ j. 

Choose j G KQ{n — 1) n [to — 1, n — 2]. We have tqjQ >~ Q, since j G [to — 
— 2] implies ||To.jQ||2n-2,2 > ||(5||2n-2.2- Differentiate both sides of (|4.28l) by 
UpeKe(n-i)\{j}i'^fiQ) - ^2n-2,p) and usc BTTfl) and gSni- Then we get 

c{TQjQ;a)= (differential of c(Q'; a)). 

Q'^ro,jQ 
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By the hypothesis of induction, c{To,jQ;a) is determined by c{Q^_i;a), and so is 
c{Ti,jQ; a) for i G [m — 1, n — 2] because of Step 2. Then we know from the resuh of 
Step 2 that c(Q+_i;a) determines the c{Q]a) for all Q e GT{X) satisfying (g^Hl)- 
Step 4. If TojQ G GT{\) for j G [1,to - 2], then the equation (|4.14p . applied to 
this J, implies 

c(TojQ;a) = ^ (differential of c((5'; a)). 

By the hypothesis of induction, c{TQ,jQ]a) is determined by c{Q^_i]a), and so is 
c{Ti,jQ;a) for z G [l,m — 1] because of Step 2. Then we know from the result of 
Step 3 that c{Q^_i;a) determines all the c{Q] a) for Q G GT(\). □ 

5. Determination of Whittaker models 

In this section, we first deduce a system of differential equations which are sat- 
isfied by c(Q;a) for Q G GT(A) satisfying and (Oil) . Secondly 
obtained are the Mellin-Barnes type integral formulas of the solutions of this sys- 
tem of equations. Though the dimension of the solution space is high, only a few of 
the solutions satisfy the whole differential-difference equations Vj^ ^(f) — 0. Lastly, 
continuous intertwining operators are determined. 

5.1. Scalar differential equations. In this subsection, we assume that Q satisfies 
(11201), (1123) and dmi). In this case, 

Ki{m') = {p G [to - l,m' - l]|g2n-3,p < q2n-2.p}, 

K2{m') = [1,to' - 1] \ ifi = [l,m - 2] U {p G [to - l,m' - l]|g2n-3,p = <?2n-2,p}, 
Ksim') = {p G [-n + 2, -TO']|g2n-3 > '72n-2,-p+i = A_p+i}, 
Ki{m') = [-n + 2, -m' - 1] \ i^s 

= {p G [-n + 2, -to' - l]|(72n-3,-p = q2n~2-p+l = A-p+l}, 

K^im') ^ {p - l\p e Ksim')} 

= {p G [-n + 1, -to' - l]\q2n-3-p-l > q2n-2-p = A-p}, 

by definition. 
Lemma 5.1. Let 

J{m') := [-n + 1, -to' - 1] U [to - 1, m' - 1], 

dm'{Q) := '2ri-2,-p + hn-S.p- 

pGJ{'m')U{m'} p€J{m') 

Suppose Q G G'r(A) sate/^es (14201) . (14^ and (gSl). 

(1) //T_„.,oQe GT(A), i/ien 

ryi a2ri-3,m'(T-m',o(3) npeJ(m')('2«-3 -m' - ^2n-3,p) 

(5.1) -t = — ^ c{T^rn'fiQ;a) 

"1 llpeJ(m')U{0}'.*2«-3 -m' + «2n-2,-pj 

= f± /'"^''" V^{Q)+Vf{Q) - d,„.(Q) +'2„-2,-™') c(Q;a). 

\ t2n-2,-m' / 
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(2) If q2n-2,m' ~ 92n-3,m' > ^m' + l, then 

(5.2) 

L (2^2ri-2,-m' + 1) JlpS J(rri') (^2n-2,-m' + ^2n-3,p) 

Bm'{lTl' ,T-m', — m'Q) JlpG ,7(m')U{0} (^2n-2, -m' ^ ^2n— 2,-p) 

X {VfiQ) - l2n-2,-m')c{T^m' -m'Q;a) 

^ hn^l^n vf{Q)+Vf{Q)~d^,{Q) - l2n-2,-m' - l] c{T^ra' flQ': o) 
'2n-2,-m' / 

m a2n-3,-m'('3)(2^2n-3,m' " 2) Hpe J(m') ('2n-3,m' " hn-3.,p ~ 1) , „ , 

+ i ff n T\ "^W' 

«1 llpGJ(m')U{0,m'}l'2n-3,m' + t2n-2,-p " ij 

(3) If q2n-2,m' = '72n-3,m' > Am' + l, then 
ai l2n-3,~m' ~ l2n-2,~m' + ^ 



(5.3) c(r„„',_,„'(5; a) = T 



Vl 0,2n-2,m' {T-m' ,-m'Q)o.2n-3,m' {'T-m' ,oQ) 
^ npe[-n,-l]u[m,n] ('2ri-2,-m' + ^2ji-l,p) 

npe[-n+l,-m+l]u[0,n-l]\{-m'}('2n-2,-nx' — hn-2,p) 

)c{Q; a). 

(4) For fc e X6(m') 
(5.4) 

t npg./(r»')U{-m'}(^2«-2,fc+/2»^3,p) ^ 7 O- «^ 

D-k{-K,To^kQ) llpgj(„/)u{0}(,'2n-2,fe -t2n-2,-pJ 
.^2n-l,n„± 



2n-2.fe 



I?2 (Q) (Q) - rfm'(Q) - '2„-2,fc c{Q-a 



Proof. (1) Since —to' e K^{'m'), the equation (|4.19p holds for z — — m'. If G 
/i(m') \ {0} = ii:i(TO') U Ksim'), then tq^-jQ ^ GT(A) because of If 
j' € [— m + 2,-1] and j € /i(to'), then Ti' j'Tq^-jQ ^ GT{X) for ah i' because of 
dHOl- If i' e [-w' + 1,-1] U [m',n- 1], then t,v,oQ ^ GT(A) because of 
and (|4.24l) . Therefore, the terms in the right hand of (|4.19p vanish except for the 
third hne and the {'D^{Q) — ?2n-2,o)c(Q; a) term in the first hne. If we arrange the 
coefficients by using hn-s.p = -^2n-2,-p for q e K2{m') and /2ri-3,p = -'2n-2 -p+i 
for q e K4{m'), we get (|5.ip . 

(2) Since To,_m'T_,„'^o<3 — T^m\-m'Q e GT(A), the equation (|4.22p . with Q 
replaced by T^m'flQ G GT(A) and with j — — m', holds. Since To.-j'T_m'^oQ ^ 
GT(A) for j' e ii'i(TO')U7^5(TO') and t,/,ot--™',oQ ^ Gr(A) for i' e [-m' + l,-TO + 
1] U [m' + 1, n — 1], the terms in the right hand of (I4.22p vanish except for the second 
line, the {Vf{Q)- hn-2,o)c{T-m' ,oQ] o) term in the first line and c{Tm,' ,oT-m' ,oQj o.) 
term in the third line. Thus we get (15. 2p . 

(3) For this Q, Ti -m'Q G GT(A) if and only if i = — m'. (|5.2p is the equation 
(14.151) applied to j = — to'. 

(4) Consider the equation (|4.22p applied to j = k. Since to^^jiQ ^ GT(A) if 
f e Ki{m') U K^im') and t,.,oQ ^ GT(A) if i' e [-to' + 1,-m + 1] U [m',n - 1], 
the terms in the right hand of (|4.22l) vanish except for the second line and the 
{'D^{Q) — l2n-2fi)c{Q\ a) term in the first line. Thus we get (|5.4p . □ 
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Proposition 5.2. If Q e GT(A) satisfies (g^, and (g^, then c{Q;a) is 

a solution of 

(5.5) l{Vf{Q)-drn'{Q)f-iVt{Q)±l2n~i..nf'(^-^^ \c{Q;a)^0. 



Proof. Suppose q2n-2.m' = 92^-3, m' > ^m'+i- Then (|5.5p is obtained from (|5.1I) . 
(j5.2[) and (|5.3p . by eliminating c(r_m',oQ;a) and c{T-m' ,-m'Q',CL). 

Next, suppose q2n-2,m' = <Z2n-3,m' = ^m'+i- The equations (15.21) and (|5.3p . with 
(5 replaced by Tj^i ^'Q, are difference equations for Q, Tm'.m'Q and tq „iiQ. The 
equation (j5.4l) . applied to fc = m', is a difference equation for Q and TQ^m'Q- By 
eliminating Tm',m'Q and tq „i'Q from these, we get the equation (j5.5p . □ 

In order to deduce another differential equation, we prepare two identities. 

Lemma 5.3. For xi, . . . , Xk, yi, . . . ,yk and z, 

i=l lli'=l,^i(2^« ^J') i=l 

(5.7) UlM + y,) ^ Ulii^-m) ^ 

»=i (^: + a;»)n!''=i,#^(a;» -a;,') ULii^ + 

Proof. (15. 6p The left hand is a polynomial in z of degree /c — 1, which we denote 
by f{z). For I G [1, fc], /(x;) = ni=/(^i + Vi)- ^he other hand, the polynomial 
YliZii^ + Vi) of degree k — 1 has the same values at k points xi, so they are identical. 

(|5.7p Let ^(z) := (left hand) x ni=i(^ + ^i); which is a polynomial in z of degree 
A: - 1. For Z e [l,fc], g{-xi) = {-l}''^'^ ULii^i + Vi)- On the other hand, the 
polynomial rii=i(^ + ^i) ~ T]!i=i{^ ~ Vi) of degree fc — 1 has the same values at k 
points z = —xi, so they are identical. □ 

Suppose Q e GT{\) satisfies 1^^, (g^Sl), (|i^ and g2n-2,m-i < In 
this case, Kg{m') is not empty since m — 1 e KQ{m'). The equation (|4.14p . applied 
to this Q and j = m — 1, is 

(5.8) 

|p2±(Q)TA„+i + ^(6Ao(0)Tl)|c(Q;a) + .^ ^ Ak{Q)c{T^,kQ:a) 



= 0. 



Let j g Ke{m'). After applying npGif6("i')\fa}(-^2 (Q) " '2r!-2,p + 1) to both sides 
of dEHl), use (g:^ and (EH)- Then we get 

(5.9) 



-^^j(<9) n (^2ri-2j-^2n-2,p) c(roj(3;a) 

\peK6(m')\b} / 

n (vfiQ) - /2„_2,p + 1) IvfiQ) T A„+i + |^(tAo(Q) T 1)| 



.pe-f!:6(™')\b} 
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keKe{m') llpG,7(„i')u{-m'}i'2n-2,fe + '2n-3,pj 



^2 (Q) ~ hn-2,k 

c{Q;a). 



X fT^^^^^2?2^(Q) (Q) - dr„'(Q) - '2„-2,fc 

V hn-2,k 

Let 

(5.10) K7(to') {pe [m- 1,™'- e Ke{m )} 

u{pe + 1,-m' - i]b e ii:6(TO')}- 

Note that #Ke{m') = ji^Kj{m') + 1 if 

'?2ri-2,m-i < ^m-i- This is bccause m — 1 G 
i4r6(TO') if (j'2n-2,m-i < Am-1, whilc there is no element in Kj{m') corresponding 
to TO — 1 e K(i{m'). It is not hard to see that 

iA [Q) - np=l ^2ri-3,p IIp^l hn-l,p iTp^l {-hn-2,p) _ hn-lji ]\p(zKj(m') hn~3,p 
rip^l ^2n-2,p(^2n-2,p — 1) 0^=1 (~^2n-l,p) YipeKeim') ^2n-2,p 

and 

A (n\ n (I. ^^ ^pgJl(m')(^2n-2,fc - ?2n-2.-p) 

llpe/2(m')'>'2n-2,fc + «2n-3,pj 

_ npe_ftr7(m')('2n-2,fc - '2n-3,p) 
Y{p^Ke{m')\{k}i^'in-2,k - hn-2,p) 

It follows that the last term in ()5.9p is ±77201/202 times 

Y[peK7i^^n-2,k — hn-Z,p) 



E 

keKeirr 



IlpeKe{m')\{k}i^2n~2,k - hn-2,p) 
IlpGKe{m')\{j}i'^2 iQ) ~ hn-2,p) " Ylp^Keim')\{j}i^'2n-2,k - hn-2.p) 



^2 (Q) ~ hn-2,k 




X ( ( 1 T ) (2?2*(Q) - l2n-2,k)+Vf{Q) - dm'iQ) - Vf {Q) T /2«-l.„ 

l'2n-2,k J 

hn-2,k T hn-l,n YlpeK7{m')(^'2n-2,k — l2n-3,p) 



hn^2,k ]\peKs(m')\{k}Q''2n-2,k - hn-2,p) 

X n p2^(Q)-^2„-2,p) 

pei<-6(m')\{i} 

^ npGX7(m')(^2n-2.fc - ^2n-3,p) npei<'8(m')\{fe} (^^^ '^'3) " ^2n-2,p) 



,■) npeK6(m')\{*;}('2n-2,fe hn-2,p) T^2 iQ) ~ ^2n-2,j 

0-dm'(O)-2?2^(Q 
{hn-2,j — hn-3,p)-. 



keKei 

(VfiQ) - d„,,{Q) ~ vtiQ) T hn-^ 

1 



peKr(™') ^2 (Q) - ^2„-2,j 
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X IT 



2n— l.n 



+ 



^2 (Q) ~ ^2n~2,j 

X (VfiQ) - d„AQ) - 'DfiQ) T ^2„-i,„) 

l2n~2,j T hn-l.ii TT /, , n 

1 11 (,'2n-2.j — '2ri-3.pj- 

'2n-2j ^-^t ,N 

'■' peKr{m') 

Here, we use Lemma 15.31 to get the last equality. Then (|5.9I) becomes the next 
equation. 

Lemma 5.4. IfQ e GT(A) safe/?es g^Ol); and (g^l, then for j e Keim'), 

(5.11) 



^7201 

2a2 



^j(<9) n (^2n^2j - ^2n-2,p) c{To,jQ; a) 



ipe-R-6(m')\b} 



n (VtiQ) - hn^2,p + 1) {Vt{Q) T A„+i)c(Q; a) 

pGK6(m')\{j} / 

Tlp(iK-r(ni')i'^2 i.Q) ~ ^2n-3,p) " ]\p(zK-,(r,%')i^'^n-2.j - ^2ri-3,p) 



± 



>?2ai 
2a2 



^2 (Q) ~ ^2ri-2j 

(Pf (Q) - d™-(Q) - 2?2^(Q) T ^2„-i, 



2n-2j" T hn~l,n 
l2n-2,j 



n 

p^K-r{m') 



2n-2j" — hn-3,i 



c{Q;a) 



Proposition 5.5. If Q e GT{X) satisfies (1424)) and q2n-2,m-i < 

Am-i, i/ien c{Q;a) is a solution of 

(5.12) 

{VtiQ) T An+l) n C^tiQ) - hn-2,k + 1) 

±'^{Vf{Q)-d^,{Q)-Vf{Q)Tl2n^Un) n p2*(Q)-^2n-3,p) 

pS-R'7(ni') 

X c{Q;a) 
= 0. 

If m' = m — 1 and q2n-2.m-i = -^m-i; i/ien c{Q]a) satisfies the equation ()5.12p 
with KqIui') replaced by KQ{m — 1) U {to — 1}. 

Proof. Suppose q2n-2,m-i < \n-i- In this case Kf,{m') is not empty since m— 1 G 
Ke{m'). Differentiate (|5lT|) by V^iQ) - l2n-2,j + 1, and use (fOj) . Then we 
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get the equation (|5.12p . Suppose m' — m ~ I and q2n-2.m-i = -^m-i- Then 
KQ{m, — l){T^rn+i,-m+iQ) = KQ{m — 1)(Q) U {to — 1}. It foUows that the equation 
(|5.11l) holds for = to — 1, if we replace Q by r^m+i.-m+iQ- From this equation 
and the equations (|5.ip and (I5.3p . applied to m' = to — 1, we obtain (|5.12p by 
eliminating c(r_,„+i^o(3; a) and c(t 

— m+1, — m+1 

Q;a). a 

5.2. Solutions of (|5.5p and (|5.12p . Suppose A e S„j -t, to = 2, . . . , n, and m' E 

[to - 1,71- 1]. Assume Q e GT(A) satisfies (g^, (|4l^ and KWi . Define and 
by 

{js, ■ ■ ■ Jni} Kfiim') n [to,?7i'], < js < ■ ■ ■ < Jni, 

{kN^+i, . . . , fcAT^} = Ke{m') f] [-n + 1, -to' - 1], kN^+i < • • • < /cat^ < 0. 

If Ke(m') n [to, to'] (resp. KQ{m') n [— n+ 1, —to' — 1]) is empty, then we set A^i = 2 
(resp. Ni). 



When to' > to, define 
Up = ap{m', Q) 



±A„±A„+i, forp=l, 

±A„ + ;2n-2,m-l - 1, for p 2, 

±A„ + l2n-2.k^~l, for iVi + 1 < p < iV2. 



Suppose to' = TO — 1. In this case, (i) Kg{m') n [to, to'] is empty, and (ii) the 
inequality relation of ±A„+i and ^2n-2,m-i ^ 1 depends on the value of (?2n-2,m-i- 
For these reasons, we define 

{±A„ + max{±A„+i, Z2n-2,m-i - 1}, for p = 1, 
±A„ + min{±A„+i, Z2n-2.m-i - 1}, for p = 2, 
±A„ + ;2«-2,fe, -1, for3<p<iV2. 

Remark 5.6. When to/ G KQ^m'), l2n-2,m' corresponds to except for the case 

to' = TO - 1 and ^2n-2,m-l " 1 > ±A„+i. 

Consider the number hn-a.p for p E Kr(m'). By definition (I5.10|l of Kr{m'), 
Kj{m') = {js - 1, . . . , jATi - 1} U {/cwi+i, . . . , fcAfJ. 

Define 

R - R (m' n\ — / + ^2«-3,jp-i, for 3 < p < TVi, 

Pp - Pp(m , C^j .- I ^ for iVi + 1 < p < N2. 

Lemma 5.7. These numbers satisfy 

ai >a2 > ^3 > "3 > ■ ■ • > "ATi-l > ^ATi > ttNi 

> > /StVi + I > OiNi + l > Pni+2 > ■ ■ ■ > aWa-l > > OtN^i 

and the difference between /3p and ap is at least two. 

Proof Recall Remark l3.10l and the condition (|4.24p . li p e Ke{m')n[m-l,m/ -1], 
then l2n-2,p - 1 = Ap+i + n - p - 1 = Ap+i since q2„-2,p = Ap+i. If p G Ke^m') n 
[-n - 1, -to' - 1], then l2n-2,p - 1 = -hn-2,-p = -(A-p + ?i +p) = -A_p since 

9211-2, -p = A_p. Wc know k^' > ^2n-2,m' - 1 > A„i' + i - 1 since Xm' > q2n-2,m' > 

A„i'+i. It follows that 

Qfl > Q!2 > ■ ■ • > CtNi > > UNi+l > ■ ■ ■ > OiN2i 
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since the Harish-Chandra parameter A = X)"^!^ ^i&i G ^m.± satisfies 

Ai > • • • > A™_1 > ±A„+i > km > ■ ■ ■> An-i > |A„|. 

Next, consider the numbers /Sj. li 3 < p < Ni, then ^2ri-2jp~i — 1 > hti-ajp-i > 

hn-2Jp > hn-2,jp " 1 sinCC g2«-2jp-l > g2n-3jp-l > g2n-2 Jp ■ If iVl + 1 < p < 
N2, then l2n-2,kp-l - 1 > '2n-3,fcp > hn-2,kj, > hn-2,kp - 1 sinCe 52n-2 -fcp > 

q2n-3,-kp > q2n-2,-kp+i- It follows that ttp-i > ;3p > Q!p, and the difference 
between /3p and ap is at feast two. □ 

In order to rewrite (|5.5p and (j5.12l) in a convenient form, iet 

(5.13) h:=^, t2:^T^, dt^-^t— {i = 1,2) and 

(5.14) n{Q,m';a) 5:^^-^^--..+!:.-^ '--,.t(a„+a„+,)-™+2 

Note that ti > since ryi > and ai > 0. We have di — —dt^ {i — 1,2) and 
7]2ai/2a2 = Tt2/ti- Since l2n-2,n = = A„, we have the foflowing proposition. 

Proposition 5.8. Assume Q e GT(A) satisfies (g^Ol), and (PTM]) . £»e/jne 

/(Q, m'; t) := ^(Q, m'; a)"^c((5; a). 

Then, the differential equations ()5.5p a^ii ()5.12p are expressed as 

(5.15) (a,2^-a2^-i2)/(Q,™';t)^o, 



(5.16) 



n(ft, +ap) + ^(ft, -ftj X{{dt2+P,) f{Q,m';t) = 0. 

.P=l P=3 J 



Proposition 5.9. Let Cj he a loop starting and ending at +oo, crossing the real 
axis at —aj — 1 < s < —aj , and encircling all poles of T{^ap ~ s), p = j, . . . , N2, 
once in the negative direction, hut none of the poles of T{j3p + s), p = . . . , j , i.e. 
encircling the half real axis {a; + Oi G C\x > —ctj} once in the negative direction. 
Define 



ds. 



f[{Q,m';t) j ■ 2niJc, Upls^ - ^ - ''^) lM2)^/-.(ti) 
and, for j — 2, . . . , N2, define 

ff{Q,m';t) 
fj{Q,m';t) 



2^^^ Jc, UirJi r(i + ap + s) Upl,+^ r(i - /?p - .) I n i-Ah) 



ds. 



Here, K^{z) and I^{z) are modified Bess el functions (cf. [4]/ Then these integrals 
ahsolutely converge for (ti, t2) G \ {{ti = 0} U {t2 = 0}), and they form a hasis 
of the solution space of the system of equations (|5.15p and (|5.16p . Moreover, if 
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Qfi ^ 0L2, the leading terms of these functions at t2 — are 



// 

f' 

J 7 



Upl2 r(ai - ap) 



nr=,+ir(a,-ap)m=3r(/3p-a,) 



3 + 



1 T{a, -Pp + 1) 



(-t2)-"^i^a,(ii) 

^2 ' laAtl) 



if j ^ 2, respectively. If ai — a2, then the leading terms of f^ , f( are complicated. 

Proof. At a generic point, the solution space of the system of equations (j5.15p and 
(|5.16p is 2N2 dimensional It is easy to verify that these integrals formally satisfy 
the equations (15.15^ and (j5.16p . 

Let s ^ u + VL, where u is a very large positive real number and t; is a non-zero 
finite real number. We will see the asymptotic behavior of the integrands when 
w — > 00. In the following of this proof, Cj(w)'s are positive constants which depend 
on V. 

By the asymptotic expansion of the gamma function, we know that, if | arg s| < tt 
and |s| is large. 



s — ^ 00. 



Hence we have 

tN2 t 



n 



r(s + a) = s"~^/2+"e-"y2^ x 0(1), 

r{-ap -s)Ui=3T{l3p + s) 



n^=3r(i-/3p-s) 



n 



N2 



an 



s)Upi,+ini-Pp-s) 



< Ci{v) exp 



N2 N2 \ 

^ ap + 2 /3p - iV2 + 1 J log I 

p=l p=3 / 



2u 



Since 



{z/2Y 



and K^{z)^l'-^^^^1^^. 

2 sm VTT 



I + log I - u - vi\ - u(log y 



1) 



and tl is a positive real number, we have 
\K^u-vi{ti)\Al-u-vL{ti)\ < C2(w)exp 
Next, 

|(±<2)"+''1 < C3iv)\t2r. 
Thus the integrals in this proposition absolutely converge. The leading terms are 
obtained by the residue theorem. They imply that /^^, j — 1,. . . ,N2, L 
are linearly independent. 



K,I, 
□ 



5.3. Solutions of the whole differential-difference equations. We consider 
whether the solutions of scalar equations obtained in the previous subsection satisfy 
the whole differential-difference equations ^(f) = 01 not. 

The equation (|5.2p . with Q replaced by Tm',m'Q, expresses c{Tm' ,m'Q', a) as a 
sum of differentials of c{Q; a) and c(ro,rn'Q; a)- The equation (15. lip , with j = m' , 
expresses c(ro,m'Q;a) as a differential of c{Q;a). By (i) eliminating c(ro,m'Q;a) 
from these equations, (ii) changing the independent variables from Oi to ti and the 
dependent variables from c{Q;a) to f{Q;a), and (iii) simplifying the equation so 
obtained by using (|5.15p and ()5.16p . we get the following shift operator. 
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Proposition 5.10. Suppose Q e GT(A) satisfies (g^Ol); and (g^ll), and 

suppose Tm',m'Q G GT{X). Then 

f {Tm' ,m' Q , "m' 7 1) = (nonzero constant) x Si{m' ,Q)f{Q,m';t), 
Si{m',Q) 

^ Tlf.^''^ + '''\JIn} ' KT^^i • 

For notational convenience, let Q' := T,n',m'Q and denote aj{Q'), l3j{Q') by a^, 
f3j, respectively. As before, aj, 13 j mean aj{Q), l3j{Q), respectively. Let A^i = 
#{Ke{m'){Q) D [m, m']) + 2 and N2 = #iKe{m'){Q) n [-n + 1, -m' - 1]) + iVi, i.e. 
they are the numbers Ni,N2 used in the previous section, defined for Q (not Q'). 

Suppose (72«-2,m' = q2n-3.m' < '72n-3,m'-l '72n-4.m'-l- Then Ke{m'){Q') ^ 

Ke{m'){Q). Therefore, 

a'j =aj, for j = 1, . . . , iVi - 1, iVi + 1, . . . , N2, 

/3j=/3j, for j-3,...,7V2. 
In this case, it is easy to verify that 

' !^{Q'.m!-t) if J <iVi, 

p=i 



^i(m', Q)//^(g, m'; t) ^^i^^,- a^, - 1) x | i^^Y^g"^,;^) jf ^. " ^yj; 



for j = 1, 2, . . . , iV2 and L = X or /. 

Suppose m! >m and g2n-2,m' = q2n-3,m' = g2n-3,m'-i = '?2n-4.m'-i- Then 
Js^6(m')(Q') = Keim'){Q) \ {m'}. It follows that 

a'j = aj, for j = 1, . . . , iVi - 1, a'j = a^+i, for j ^ Ni, . . . , N2 - I, 

for J =3,...,7Vi-l. /3;=/3j+i, for j = TVi, . . . , iVa - 1. 

In this case, for L = K,I, 

Si{m,Q)fj {Q,m ;t) = |J_(/3p - aN^ - 1) ^ < 



0=1 



On the other hand, the non-zero functions Si{m' ,Q)f^_^{Q,m';t), L = K,I, are 

not solutions of (|5.15p and (j5.16p . Since (rm',m')''^"~''''"'~'"^"'+'Qm' = Qm'i '^'^ 
get the following proposition. 

Proposition 5.11. For m' > m and L = K, I, 

n Mm\r^,,^^,Q)f^{Q-„m';t) 

(non-zero constant) x f^{Q^,,m';t), if j < Ni, 

not a solution of (|5.15p and (|5.16p for Q^, if j — Ni, 
(non-zero constant) x fj'_-^{Q^^,,m';t), if j > Ni. 
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Suppose ml >m and Am' > 9271-4, m'-i- Consider Q G GT(A) satisfying (|4.20l) . 
and 



(5.17) 



g2n-2,p = Ap+i for p e [to - 1, to' - 1], 

q2n-3,p = q2n-4,p for p € [to - 1, to' - 2], 

q2n-2,p = Ap for p e [to' + 1, n - 1], 

92n-3,p = 92n-4,p-l for p G [to', ft - 1], 
92n-2,m' = 92n-3,m'-l G ['?2ji-4,m'-l 7 A„i' ] 



Note that, if g2n-2,m' = q2n-3,m'-i = q2n-i,m'-i, then such Q is defined in 
Definition \£22i If q2n-2,m' = 92n-3,m'-i = A„i', then such Q is 

For Q € GT(A) satisfying (|4:20)) . (14723)) and (ISTT)) . the equation (|4l^ . with 
j = m', is 

c('r,„/_i.,„/(3; a) = (nonzero constant) x —{'Dt{Q) + l2n~2 m')c{Q] a). 

m 

As before, let 

f{Qi,',rn';t) = n(Q+,, to', a)"^c(Q,+ ,; a), 

f{Q^'-i-m' - l;t) =n(Q„,_i,m'- l,ay^c{Q:^,_^;a). 

Since (rm'-i.m')'^'"' '^^""" '"'^^Qm' = Qm'-i' '^'2 obtain a shift operator 
f{Q^,_^,m' - l;t) = (non-zero constant) x S2{m')f{Q^,,m';t), 

^2(m'):= n (a*, ±A„-(?-n + TO'). 

Recall that, for 

Kfi{m){Ql^,) {m - 1} U {j3, . . . , jati} U {kN^+i,- ■ ■ , fcwj, 
KT{m'){Q^^,) is defined to be 

^7(™')(0m') = {J3 - 1, ■ • • , jWi - 1} U {fcATj + i, . . . , fcAfJ. 

Suppose g-,_i ^ Q+,. Since X6(to' - l)(Qm,_i) = Ke{m'){Q+,) U {-to'}, we 
have 

Ke{m' - l)((5m,_i) = {to - 1} U {ja, . . .JnJ U {-to', fcAr^+i, . . . ,fcArJ, 

iC7(TO' - l)((9;;,_i) = {j3 - 1,- • - liWi - 1} U {-TO',fcAr^ + i, . . . ,fcArJ. 

For simplicity, denote aj(Q+,), ^j(Q+,) by a^, and aj(Q„/_i), /3j(Q„,_i) by 
a", /3", respectively. They are related as follows: 

a'^^a'p forl<p<7Vi, l3p ^ for3<p<iVi, 

a^^^l = ±A„ - A,„' - n + to' /^TVi+I = ^ 92n-4,m'-l - ?T- + m' + 1 

a;' = a;_i for7Vi + 2<p<7V2 + l, = /J^-i for iVi + 2 < p < TVa + 1, 
and they are ordered as follows: 

a'l > "2 > /^3 > "3 > • • • > "Wi-l > P'ni > o^'ni 
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Since 

S2{m'){±t2Y 



r(i + + .) ™ ^ > r(i - p';,^^, s) ™ ' 

we have the foUowing proposition- 
Proposition 5.12. For K or I, S'2(m')//(Q+, , m'; t) is a non-zero constant 
multiple of 



(1) */j = i,...,A^] 



(2) //+i(g,-,_i,m'-l;t) tfj=Ni + l,...,N2. 

Propositions 15. Ill 15.121 enable us to judge whether fj',j — l,...,N2, L — K,I, 
generates the whole solution of I?^^ ^0 = 0. 

By Theorem 14.231 c{Q^_^;a), therefore f{Q^_i]t), determines all the c{Q;a) 
containing the same qi,...,q2n-4 parts as Qn-i- Let aj — aj{Q^_i). This is 
the leading exponent of //'(Q^t-i! ^2 — 0. Since ai — ±(A„ + A„+i) and 
a2 = ±A„ + l2n-2,m-i — 1, alternative use of Proposition 15 . 1 II and Proposition l5.12l 
implies that, if j ~ 1,2 and L = K,I, 

ff{Qt_„n-l;t) P™J^//(Q;_,,„-2;t) ^'^-^ ffiQ^.n - 2,t) ^ . . . 



...""-5^//(Q-_„™-l;t) 

On the other hand, if j > 3, there exists m' € KQ{n — l){Qn-i) ^ [m,n — 2] such 
that aj ((5^_J = ±A„ + ?2n-2,m' - 1- In this case, 

f^{QU,n~l;t) ^™-i^¥^//(Q„-_„n-2;t) ^™-e¥^/^(g+_„ n - 2; t) ^ . . . 



. . . ''''^^//(Q„,,™';i) '''■^^not a solution of ^EM and for Q^n'- 

This implies that only f^{Ql^_^,n — l;t), j = 1,2, L = iiT, /, can generate a 
solution of the whole differential-difference equations. It follows that the constant 
C in Theorem l3.8l is at most four. In the special case when m = n, we can check the 
compatibility of the equations in Lemma 14.131 Therefore, this constant C, which 
is independent of m, is just four. 

Theorem 5.13. Let A £ 'E.„i^±, m — 2, . . . ,n. The functions ff^ {Q^_-j^,n — l;t), 
fl{Qt-i, n - l ;t), f2{Qt^i,n~ l;t), l;t), with Q+_-^ defined in 

Definition \4-. 22\ completely determines the non-zero solutions ofD-^ = 0. 

5.4. Continuous Whittaker models. So far, we have investigated the space 
IIomgjfj,(7r^, C°°(G'r/A'k; 7])). Here, we specify the subspace of continuous inter- 
twining operators IIomQjj((7r^)oo, C'°°(Gr/A'k; r])). Note that the latter space is 
isomorphic to Wh'^^(7r^). 

Proposition 5.14. Suppose Gr = Spin{2n,2). Let tta be the discrete series rep- 
resentation with the Harish- Chandra parameter A. 

(1) Suppose A e S„j + and A' e S.^ - , m = 2, . . . , n. // Wh'^^(7r^) ^ {0}, then 

wh-«,) = {o}. 

(2) Let rj and rj' be non-degenerate unitary characters defined as in (|4.6p . Sup- 
pose 7nV2 < 0. // Wh!°^(7rX) ^ {0}, then Wh!?^,(7rX) = {0}. 
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(3) Suppose A £ Ern,±, m = 2,...,n. IfWh°°^j{TTl) ^ {0}, then 
dimWn „(7rA)= > dim ' , , s. 

-^m>A^l>-^m + l>■■■>-^7^-l>MJ^_^>l•^T^ 

Proof. For our case Gr = Spin{2n,2), there are two principal nilpotent GR-orbits 
on 00, and Wg^ ~ 62 k (Z/2Z)2. Therefore, (3) follows from Theorems El [SH 
Remark 13.101 and Theorem 15.131 

Via the Kostant-Sekiguchi correspondence ([T^), these two orbits correspond to 
the two nilpotent ii'-orbits on p generated by X„ei±e„+i + ^e„^e„+i + ^-e„=Fe„+i 
(see the proof of Proposition 13. 5p . Schmid and Vilonen ([H]) proved that the 
associated cycle of a Harish-Chandra (g, iirR)-module and the wave front cycle of 
it are related under the Kostant-Sekiguchi correspondence. It follows that, for 
m — 2, . . . ,n, the wave front set of the discrete series tta with A G Sm,+ and that 
of TTA' with A' S Sm,- are different principal nilpotent GR-orbits. Therefore, (1) is 
a consequence of Theorem 13.21 fl). Recall the identification of a unitary character 
with an element of t(no/[no, no])* C lQq ~ lqo (cf. ^'S.l^ . It is easy to check that 
r] and ry' are contained in different principal nilpotent GR-orbits multiplied by l. 
Therefore, (2) also follows from Theorem 13.21 (1). □ 

By a theorem of Wallach ([H]), if V e Homg;((7rX)oo, G°°(GR/iVR; 77)), then 
ip{v)(g), V £ (7r^)oo, g G Gr, must be a moderate-growth function. We show that, 
if T^2 > 0, then the function defined in Proposition 15.91 generates a rapidly 
decreasing Whittaker function. 

Recall the definition (I5.13P of ^1,^2- Since 01,02 > 0, 771 > and =F'72 > 0, we 
have ti > Q and t2 > 0. Let 

^2 -, „ 

^3 = n n (-^*^ ~ '^)' •/'o W = ^ / ^(""1 ~ s)r(~a2 - s)tf, K^sih) ds. 

p=3q=ap + l JCi 

Then ff^ — S^fo, so we show that /o is a rapidly decreasing function. Recall an 
integral formula 



1 /zx'^ f°° / z 



^^(")^2y io '"pr""i^J"" 



v2 



of is:^(2). Then /o is 








-I— S)(i^X/'---»'^'---»)(^)"-)" 



By residue calculus, the inner integral is expressed by a if-Bessel function, and 
then we get 



00 







tl \ /2t2w\"^"'+"'^^' I2t2u\ du 



fo= CXp -U - — — — Ka,-C2 2 



Au / \ ti / \ \ ti J u 



This is essentially the same as the function hrfi treated in [6l Theorem 4.4]. This 
function is proved to be a rapidly decreasing function there. It is not hard to see 
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that this function generates a rapidly decreasing solution of V-^ — 0. There- 
fore, the intertwining operator corresponding to this solution is an element of 
HomQjj((7rX)oo, C°°(G'r/A'^r; ??))■ This result, together with Proposition 15.141 im- 
plies the following theorem. 

Theorem 5.15. Suppose Gr — Spin{2n,2) . Let tta be the discrete series repre- 
sentation with the Harish- Chandra parameter A € S,„ j-, to = 2, . . . , n. //T^ > 0? 
then 



dimHomg; ((^X)oo, G°°(Gr/A^r; 77)) 

(Aii,...,/j„_2.A'i, 



dim V(^i^ 



Each continuous intertwining operator corresponds to defined in Theorem ] 5. 91 
On the other hand, if ^rj2 < 0, then 

Hom- (K)^,C-(G«/iVM;7?)) = {0}. 
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